UDN
Search public documentation:

PackagesAndNetworking
日本語訳
中国翻译
한국어

Interested in the Unreal Engine?
Visit the Unreal Technology site.

Looking for jobs and company info?
Check out the Epic games site.

Questions about support via UDN?
Contact the UDN Staff

UE3 Home > Networking & Replication > High-Level Breakdown of Packages and Networking

High-Level Breakdown of Packages and Networking


Overview


If you've done any editing of the Unreal Engine, you'll know what a package is. All game content is stored in packages: textures, maps, code, etc. A package, is nothing more than a collection of "stuff". A package can contain a map, textures, code, sounds, music, or any combination thereof. The directory structure of Unreal is just for our feeble human minds to organize things into separate places for textures, music, and maps. So a mapper can include textures and code in his map, removing the need for including .u files along with the map. In general, however, it's best to stay with the example set by the Unreal Engine, for sanity's sake. Put maps in .umap (or your own specific map extension), code in .u and the rest in .upk in organized directories. One thing you might want to do to improve Unrealscript compilation performance is to split up the package. Don't include game textures and models in your code packages. Instead, compile these into packages that contain a dummy class. This allows your art data to be compiled once beforehand, and your code recompilation merely references those packages (though the #obj load exec line). Another advantage is that any patches can be distributed as the smaller code file, avoiding a re-download of the entire package that includes models and textures.

ServerPackages


When the server starts up, it loads all the packages it needs for the game. It loads the map, its associated textures, sounds, and music packs. It loads the game, as defined in ?Game=XX in the ServerTravel command. It loads the severpackages mentioned in the gamename.ini (ExampleGame.ini). Once these packages and classes are loaded, no further packages are added to this list of packages stored in the server. This list, which we will call the ServerPackages list, is sent to every client that joins the server. The client then loads these packages on their own, so that any actors sent to the client will be able to be loaded without any hassle. No code executes until after all these packages have been loaded. If this is a server, it will not accept any clients until these packages have been loaded. This explains the long non-responsive delay when you load levels or servers in Unreal. Additional packages can be loaded through the use of DynamicLoadObject() (ALERT! Note that DynamicLoadObject() does not work on all platforms!), but these packages are loaded on the local machine only. Unreal does not tell any connecting machines to load these packages as well. All packages required for a level to load are automatically added to the package map.

This explains a few strange phenomenon that people experience. Mutators are one such example, and it's useful to read this even if you have no interest in mutators, since the problems can manifest themselves in other areas as well. Mutators are loaded by UnrealScript using DynamicLoadObject(), specifically in GameInfo::InitGame(). This happens after the list of packages have been loaded, so mutator packages are not loaded into the ServerPackages at this point. The mutators included with Unreal do not have this problem because they are included in the game code itself. Any mutators which operate server-side only are fine, since they don't require sending any actors to the client in that package. If however, a mutator has actors defined which it sends to the client, then its actors will be invisible on the client. Because the package was never added to the ServerPackages list, the client never loaded the package, and so the client has no idea of the actors defaultproperties, and it is invisible. The way to solve the problem is to explicitly put the problematic mutator into the ServerPackages array in the GameName.ini file, under [Engine.GameEngine]. This causes the mutator package to get loaded before the game begins, and then when the GameInfo loads it, it is already in the ServerPackages and so will work fine on the client.

The same applies to skins packages. The user's skin is specified in the joining commandline options, and it is not loaded until the Login() in GameInfo. Again, this is after all ServerPackages have been loaded. So every client will get this texture replicated to them, but when they go to display it, the texture package is not in the client's local memory (since it was not in the ServerPackages), and so you see the infamous green skin problem.

GUIDs


Every package has a GUID associated with it, which stands for Globally Unique Identifier. Every package has a GUID generated for it when it is saved. This GUID can then serve to identify different version of the same package, or to differentiate between packages. The GUID takes into account the time and other information in generating a 128-bit ID, so you can be sure that no two GUID's will ever be alike through pure chance. There are 2128 = 2.4 x 1038 possible GUIDs. I'm sure that's enough to justify not having to worry about the problem.

In the process of conforming a package, the new package is given the same GUID as the original package, among other interesting details. They are deemed to be alike for all intents and purposes. We'll see why this in a later section.

Connecting to a Server


When you connect to a server, the process roughly goes like this:

  1. The server sends ServerPackage list, along with GUIDs for each package.
  2. The client checks the list in order, trying each download manager until it finds one that it actually has on the client.
  3. The server checks to see if it this method is allowed, and if so, how. If the user requests Engine.ChannelDownload:
    1. The server checks to see if the [IpDrv.TcpNetDriver]'s AllowDownloads boolean is true, and doesn't send if it is.
    2. The server sends the file to the client. This may cause lag for other clients as the bandwidth is spent sending a file to that particular client. The maximum speed at which the file is sent is determined by [IpDrv.TcpNetDriver] -> MaxClientRate.
  4. If the client cannot download, because the server will not allow the client to download packages, then the client gives up, and fails to connect to the server.

Conformed Packages


A conformed package is intended to provide network-compatibility between two versions of a package. The way it works is this: Say I release version 1 of my product, and now want to release version 2. If I don't want this new version to prevent unpatched users from playing on patched servers, nor do I want it to prevent upgraded users from playing unpatched servers, then I have to use version conforming. Version conforming lets Unreal pretend the two versions are the same, and allows users of the original package to play with users of the upgraded version. Package conforming requires the original version you wish to conform against, obviously, so it knows how to tell the new version to 'make-believe' that it is the old one. There are two ways to conform a package:

  • Put a copy of the original in a guires directory in your game directory. It should be next to your Binaries directory. Any versions compiled with 'ucc make' will be automatically conformed against the version in guires.
  • Compile a copy of the new version of your package. Then use the conform commandlet to conform the package.

When you compile a package, it includes an array in the package called Generations, that stores a history of the conformity of the package. So when a server with version 2 sees a client of version 1, it looks up version 1 in the Generations array and knows how to communicate properly with that client.

This has implications for conforming as well. When you want to conform version 3 of your package, in order for it to conform to both version 1 and version 2, you must conform version 3 against version 2, not version 1. Version 2 contains information on how to communicate with version 1 and version 2. Since version 3 needs this information to be compatible with all of them, it must be conformed against version 2. So whenever you release a public version of your code that will be outside of your control and possibly publicly released, it's best to put that version in your guires directory to ensure that the next version you compile internally (and eventually release publicly) will be compatible with all versions out there. Each version of your package must be compiled against the previous one, not the first one. So when a client of version 4 sees a server of version 2, it will know how to be backwards-compatible and communicate with the unpatched server.

The conforming process actually consists of keeping the name tables in synchronization. When version 2 is conformed against version 1, it saves the name table, keeping all names that appeared in version 1 in the same position. Any additional names that appear in version 2 show up after this list. When the server and client send information about functions or variables, they actually send the index into the name table. So by ensuring that the name table used to communicate is identical even if the client is version 2 and the server is version 1, it allows communication to occur flawlessly. When storing the generations array to disk, it stores the number of names and number of exports. This lets Unreal know that a version 2 package can communicate with a version 1 package using the first X entries (which are exactly the same as version 1) of the name table.

When conforming Localized packages, you want to conform them in a chain so that each language with work with every other language. For example, if you had 'int', 'fra', 'esn' and 'deu' localized versions of a package, you would want to conform 'int' to 'fra', then 'fra' to 'esn', then 'esn' to 'deu', etc. The order of the languages does not matter.

Package-flags warning


There are a few package flags that you will find interesting and necessary. If you want to use your package in net play and allow it to be downloaded to the client, then you must turn on the AllowDownload flag for the package.

Package Mismatch Errors


Downloading a package with the same name but a different GUID will result in a package mismatch error. This can be resolved by conforming the package with the previous version of that package (and thus creating a new generation). However, the anticheat protection will kick in if the checksum of both packages isn't know to the server.