GPG: You must implement automatic custom map downloading
One of the largest deficiencies in Supreme Commander custom multiplayer right now that will only be made even more obvious with the release of the new patch is the lack of automatic transmission of custom game elements (such as units, maps, and mods) between players. This lack means that, in order to play a custom game, all of the players must go to an external website (like SupCom Source) or transfer protocol (like sending files via AIM) to get the customizations, install them, and then get together on GPGnet and play. There is no way to transfer the customizations in GPGnet as of yet. And if you’re looking at a custom game that you want to play but you lack the appropriate map, mod, unit, whatever, you have to basically resort to Google to try and find it, install it, and join the custom game before the host launches or gives up in frustration because no one is able to join.
Gas Powered Games, you need to get with the times. StarCraft had this functionality out of the box when it was released nine years ago. There’s utterly no excuse for it to not be in a current game. With the new patch the kinds of things modders will be able to do with custom maps will be amazing. It will exceed anything StarCraft or WarCraft III were capable of. But if the players cannot even download these new custom maps on-the-fly through GPGnet, how are they supposed to play them?! The inconvenience factor of having to download everything externally is hugely limiting.
The one “excuse” I have heard is that 3rd-party customizations, maps especially, are large, and that it would take a long time to transfer them through the game. Big deal. I remember back in my StarCraft days playing a Use Map Settings map called Legends Open RPG. It was over 2MB and took forever to download (and this was back when a lot of people had dial-up). Yet a lot of people still played it. How? Updates to it came in large major version updates, so you could download it once and then play that version for a whole month. And people, out of the goodness of their hearts, would actually host games just so that others could join, download the map, and then leave. I remember doing that myself many times, because I loved the map and I wanted other people to be able to play it with me. What’s it matter if it takes ten minutes to download a map if you only need to download it once and then you get hours of enjoyment out of it?
But GPG actually has a better alternative they can implement that I will outline here. Each GPGnet client would maintain a local database of hashes and modification times of every file in the maps and mods directories. When GPGnet starts up, it would check the modification times of all of these files, and if any of the times change, it then checks the hashes to see if those have changed as well. Any file whose hash has changed, or any newly added file, is automatically uploaded to GPGnet in the background (provided, of course, that GPGnet doesn’t already have it, as identified based on hash values). This will not result in a lot of traffic. Really, the only people who will be uploading will be content creators, because everyone else will not need to upload since GPGnet will already have it (it only ever needs to be uploaded once). A standard hash algorithm, like MD5 or SHA1, would more than suffice.
And then, from here on out, whenever you join a custom game and it is using a custom map, mod, or unit that you do not have, it would be automatically downloaded directly from GPGnet’s servers to your computer. This solves the speed issues. Most gamers are playing on asymmetric DSL or cable lines, which means that their download speeds are very good, but their upload speeds (including to other players) are not. But GPGnet’s servers aren’t limited by asymmetric connections, so customizations would download very quickly and you would be able to join any custom game on GPGnet and automatically download all of the required customizations quickly.
The only excuse GPG has for not implementing this is that it would use bandwidth, but let’s face it, bandwidth is cheap these days, and the added utility to the game in the form of automatic downloads of customizations is worth it. Just look at how large the StarCraft and WarCraft III Use Map Settings communities became. If GPG does it correctly with Supreme Commander, word will spread, and Supreme Commander will attract a lot of larger player base of fresh players eager for mods.
I have not yet been able to confirm whether GPG is working on automatic downloading for the next patch, but I sure hope something like this is in there, because if it’s not, most of the new functionality of their custom maps scripting and map editor will be for naught.
May 16th, 2007 at 2:25 pm
SCMP_29, betrayal ocean, is 100 megabytes. I have high speed internet, yet when I use torrents (zomg torrentz) I have to limit the up bandwidth it uses to 35kilobytes so that it doesn’t take 100% of my up bandwidth. Almost all map transfer systems use P2P transfers.
This is not to say they shouldn’t put automatic downloading in. A 5×5 map is much smaller (as map size increases the file size increases exponentially) and could be transfered quite quickly. I’m just saying why they haven’t put automatic transfers in yet.
May 16th, 2007 at 3:27 pm
Actually, I don’t think file size increases exponentially. It increases with the square of the map dimension, or linearly with the area of the map (i.e. a 20×20 map has four times as much area as a 10×10 so it should be four times the file size). Neither of these are exponential. An exponential increase would be some function on the map size like 2^x, but this doesn’t happen.
Sorry for math-geeking out there for a second; I just had to point out that your use of the word exponentially was incorrect :-)
But anyway, as for the size of the map files, 100MB is huge. How compressible is it? Because obviously compression would be the first step, and since GPGnet is already using gzip to transfer compressed map replay files, for instance, they wouldn’t even have to add any real new code. If compression doesn’t get 100MB down to something a lot more manageable, you’re right, peer-to-peer transfers just won’t work very well, because most gamers are on asymmetric connections that suck for uploading. Thus my suggestion of having GPG’s servers handle map distribution. It’s really the only way to get acceptable performance for a feature that is basically mandatory.
May 16th, 2007 at 3:32 pm
I’d like if they had a pre-launch game room. The custom games list is constantly changing and you have to keep refreshing to see if a game opens with players available. You double click the room, wait 10 seconds. The game launches. You wait another 10 seconds, then you get to the match room. You find two people got in before you and the game is closed. Or that the other players have very high pings or no pings so you have to leave..
They could make a GPG client room where you could talk to people. Download the map. Exchange mods. It’s a bit hard to go doing things like that once you’re launched. People have to start tabbing in and out of it. It’s messy.
May 16th, 2007 at 3:34 pm
Molloy: Absolutely, the custom game lobby needs to be within the GPGnet client, not within the Supreme Commander executable itself. I can’t tell you how many times I’ve joined a game that was 3/4 only to be the fifth man out once SupCom finally finished launching. And it’s just really annoying having to go back and forth between the two clients. It’s very inefficient. Plus, there’s no way to know what mods are enabled (like double resources, bleagh) until you’re in the SupCom game lobby, which takes about ten seconds to load, as you said.
May 16th, 2007 at 3:45 pm
Well people would have a fair idea in advance how much they would need to download (or possibly at least warn them, “Do you want to download 150Mb of mods?”). If you are going into a custom game lobby for a 81X81 map you don’t have you would have a fair idea of what you are getting yourself in for.
Also it is an exponential increase, not the worst kind (x^n that scares the bejeesus out of compsci heads) but it’s still an exponential increase involving the input term.
May 16th, 2007 at 3:49 pm
niall: I’m not looking at the input as the length of the string representing the number, but rather, as the number itself. So double the increase and you quadruple the output — pure quadratic, which is polynomial, not exponential. It doesn’t really make sense to look at the length of the input string in this context because, at least in base 10, it doesn’t exceed two digits in any case anyway.
May 16th, 2007 at 3:51 pm
Cyde Weys: regarding math geekiness, how would you technically describe the map file size growth, given that it increases with the square of the map dimension? Is that called geometric growth? It’s not linear, that’s for sure, it must have some term if it’s not exponential. Just wondering.
May 16th, 2007 at 3:54 pm
Baddox: That’s just called polynomial growth. In the specific case where the largest exponent is 2 then it could be called quadratic growth. Polynomial and exponential are two completely different complexity classes though: 1.00001^x will still grow infinitely larger than x^10000000 for a large enough x. It’s the whole P versus NP thing.
May 16th, 2007 at 4:02 pm
And that’s why humanity WILL run out of food if we’re not destroyed by something else first. :-)
May 16th, 2007 at 4:33 pm
Ummm, isn’t GPG releasing the map vault very soon? That will be one huge step towards what you’re talking about here, and in some ways it might be better. In Starcraft, you basically only got a new map when you joined a game which was using a new map. But with the map vault, you will have a huge selection of maps to choose from before joining a game.
May 16th, 2007 at 4:46 pm
RDon: It sounds like the map vault will go a long way towards addressing the issues I brought up. All they have to do is add a hook into the map vault such that when you join a custom game that you don’t have the map for, it downloads automatically. However, the map vault does not address the issue of custom units or mods.
May 16th, 2007 at 4:49 pm
x^1.00001 will be larger than x^1000000 for large enough values of x? How does that work? And how does that relate to P/NP?
May 16th, 2007 at 4:59 pm
Will: You’re right of course, I typoed in my comment. I’ve fixed it to 1.00001^x, which is what I originally intended.
May 16th, 2007 at 5:08 pm
And as for the P vs NP thing, it’s only tangentially related, but x^1000000 could be, say, the computation time required to solve a problem that is in P, whereas 1.00001^x would be something like the computation time to solve a problem that is in NP.
May 16th, 2007 at 10:42 pm
The Unreal Engine uses HTTP redirects and special compression to send files to the user if they do not have them. This is a little bit easier to accomplish due to there being dedicated servers for the game(s). Essentially, you just set up an HTTP server with the Unreal-compressed files (.uz2) and then set a variable in your dedicated server’s INI to point downloads to that server. HTTP is much faster for downloads than the protocol Unreal uses to send data and everything is compressed (to a degree, at least, considering textures).
Given that SupCom is P2P it will certainly require a lot of effort from GPG to get mods and maps to download automatically. Sure, it’s possible, but is it plausible within whatever budget they have? Additionally, won’t this make everyone else wait why some guy downloads all the files? Just by that last part alone I don’t see it happening. The Map Vault will probably cover the basics well enough that we won’t need anything else.
And FYI, SCMP_29 compressed to 24.4 MB with a regular ZIP. RAR or 7z would be even more. SCMP_30 is 169 MB.
May 17th, 2007 at 3:50 am
All this math aside, I agree with your point entirely and think it’s a great idea! GOG does seem to be a bit behind the times for a company who’s made such an advanced RTS game :\
May 17th, 2007 at 4:31 am
There has been a topic about this on the official boards with a reply from squidbot, it’s old and I really don’t have time to wade through his other 610 posts to search for more answers but it gives a clue that GPG knows about it: http://forums.gaspowered.com/viewtopic.php?p=869
May 17th, 2007 at 4:38 am
Totally agree with this, we tried to play some custom maps last lan but after trying to shuffle the maps around to everyone we gave up and just played a standard map.
Not to mention supcom source that has the worlds worst registration system so I couldn’t even get hold of the map I wanted as it took 2 days to email me acceptance.
May 17th, 2007 at 5:22 am
Cyde Weys: You are of course right :)
May 17th, 2007 at 11:23 am
I was hoping to find out at what number those become the same size, but my calculator locks up every time I try and I really don’t feel like letting it spend the next two day’s of processor cycles on it.
May 17th, 2007 at 11:43 am
Will: those are (obviously) very big numbers. If you really care, try something smaller. It’s also easier to see graphicly. If you have a graphing calculator try graphing y1 = e^x and y2 = x^2.
For your reading pleasure check out: http://en.wikipedia.org/wiki/Big_O_notation