Setting Up A Sandbox Shard with Dataserver

How to create a SANDBOX SHARD with Dataserver

For Non-Programmers

License: GNU GPL

by Phoenix Rising version 0.01 August 2011

Shorah Explorers

AIM: The aim of this wiki is to teach a writer/builder/developer how to build a sandbox shard to test CWE compatible ages.

The aim of this document is to detail as closely as possible the exact steps to create your own Sandbox Shard using GoW's CWE Client and Zrax's Dirtsand Server. This wiki page is specifically for writers and age builders who want to test MOULa compatible ages with Dataserving capability.

We have permission from Zrax to use his README in this wiki but with a much expanded and detailed version due to to a few subleties that were undocumented. We have chosen to use GoW's CWE Client since it has been heavily updated so that it may be compiled easily and with a much more recent version of DirectX SDK and other tweaks such as reduction of lag by Tsar Hoikas. This simply means that it has better support and hopefully better gameplay on your fancy new computer!

IMPORTANT NOTE: This is a work in progress and gets highly technical, and as such is not recommended for newbies. Changes may happen along the way so if you want to add to this wiki, please be nice! We want this to appeal to every Non-Programmer young and old and have a really cool wiki to show for it. If you have helpful suggestions feel free to add them, but kindly refrain from deleting any sections. A lot of hard work has gone into documenting this procedure for the sake of all the Uru fans out there. At the end of the day all this effort is due to teamwork, since no-one can do such a mammoth task alone!

HARDWARE PREREQUISITES: For that at least two computers are needed (one of which must have an able graphics card).

As a benchmark we will be building the sandbox based on:

• CLIENT MACHINE: A Dell Inspiron running Windows 7 64 bit, with a 1Gb Ati Graphics card, and
• SERVER MACHINE: An Asus Eeepc mini laptop running Ubuntu 10.10 (Maverick).

The specs are not crucial, except for the graphics card on the Windows Machine and the Flavour of Linux on the SERVER. We have chosen Ubuntu 10.10 because although we still have to configure stuff, we dont like configuration nightmares!

A simple test to see if your Windows CLIENT can handle all of CWE's graphics requirements is to ramp up all your Graphics settings to maximum when you run the launcher for TPOTS and see if your computer runs smoothly or has screen jitter/stutter. If your machine can handle these high settings you will be less restricted in your age building later on.

NETWORK CONNECTION: This wiki will attempt to document one of the two approaches to network connectivity.

1. SANDBOX MODE - SERVER and CLIENT on a local hub using DHCP or fixed IP.
2. SHARD MODE - SERVER on dynamic DNS or fixed IP, and CLIENT accessing through the internet.

SHARD MODE will be covered in another wiki at a later date.

Both configurations have their merits. The first provides the "offline" sandbox mode for creating your own ages and testing the content, even though you will still be connecting via a network of some kind. The second makes your shard accessible to other players over the internet. This you will use when you are ready to go live!

A Little History

Cyan has come a long way since the release of Myst. Their original 2D "point and click" game eventually evolved into a fully three dimensional gameplay with unrestricted movement and engaging avatar animations. Although Myst was the forefather of MOUL, in our wiki we wont be covering the MYST engine or how to set it up. It is too outdated. If you really want to play Myst you can use your original Myst CD content, Drizzle and TPOTS.

We need to draw a line as to what versions of Cyan's engine we use, and what purpose each version serves. Below is a list of the commonly accepted Versions.

Only one version will be used to setup our SANDBOX SHARD. The CWE Client using the MOULa content!

• Choru - Cyan's Closed Beta.
• Ubiru - Ubisoft's Closed Beta.
• ABM - Ages Beyond Myst - The vanilla single player version of Uru.
• Prologue - The original MMO version of Uru. (Essentially an Open Beta)
• UruUpdate12 - Update package to fix bug with ATI Graphics cards.
• DNI - To D'ni Expansion Pack - an add-on extension that adds some of the Prologue Ages along with a recap of the Prologue story.
• TPOTS - The Path of the Shell Expansion Pack - adds even more content to ABM.
• CC - Complete Chronicles - a compilation of the first 4 packages as a single distribution.
• UU - Untìl Uru - A no longer officially supported online version of ABM.
• D'mala - The Cyan Worlds Until Uru Shard.
• MOUL - Myst Online: Uru Live - The Third Online Incarnation of Uru
• MOULagain / MOULa - Myst Online: Uru Live Again - Resurrected version of MOUL

and finally the Open Source Client that has been given to the community by Cyan Worlds, namely:

• CWE - CyanWorlds.com Engine - the recently released Open Source client based on MOUL - Note: not a full source release.

If you want to start building ages, we do not recommend following this wiki beyond the following few lines. There are many tutorials on the Guild of Writers to help you build your first age.

Assumptions and Caveats to follow this Wiki

• You will be compiling the CWE client on a WinXp 32 Bit machine.
• You know how to use Windows command line
• You know how to use a Linux command line
• You know how to compile C++ software on Windows
• You know how to compile C++ software on Linux
• You are fluent using both systems and understand the subtleties of each, such as: case sensitivity, backslash versus forward slash, and what a terminal is.
• You know how to use scalc or excel

If you fail to do any of the above, we really suggest that you do not attempt this.

The major caveat is that will have have to create some spreadsheets that are used as a template to create batch and script files. As tedious as it may seem this is due to a few issues;

• The official plasma tools does not support recursion.
• Your newly compiled plPythonPack does not support recursion.
• If you had to encrypt one file at a time you would be here for two weeks. Just take a look at how many files are in the 'SDL', 'age' and 'dat' folders!
• The scripts give you a way to update bulk content packages aimed at the content YOU want on YOUR shard.
• Once the scripts are created, you wont have to create them again, just remember to save them somewhere safe.

Ok, let's begin:

As this wiki unfolds you will notice that we will have to do work on both the SERVER and the CLIENT at any one time, so be prepared for it before it happens. We will refer to the client as Windows, and the server as Ubuntu for ease of reference.

Github and the branches

There are 3 main projects that we will pull from github.

1. https://github.com/H-uru/Plasma
2. https://github.com/H-uru/dirtsand
3. https://github.com/H-uru/moul-scripts/tree/python27

An easy way to get the code for each is to click the "Downloads" button to the far right of "branches" tab.

You will also need to download the windows version of plasma tools to help you encrypt the various files later.

• http://libhsplasma.googlecode.com/files/Plasma_Win32_r801.zip

(Note, we won't be covering how to build a Cyan compatible engine, since Cyans MOUL still runs on Python 2.3 and CWE runs on Python 2.7)

On Windows - Let's compile CWE

Our starting point is to build the CWE client based on the "master" branch on github since it is a relatively simple process and will help you get your feet wet.

To build the client you can follow the GoW tutorial and video clip below:

• Tut: http://www.guildofwriters.com/wiki/Development:CyanWorlds.com_Engine
• Clip: http://www.youtube.com/watch?v=HZbMf5W67UI

There is no need to rewrite such a detailed and well written tutorial. The guys who created this deserve a big thank you from all the uru fans out there.

Once you are done, you will have 3 files in the

.\Plasma\build\Sources\Plasma\Apps\

directory, which you will use later.

.\plClient\debug\plClient.exe
.\plUruLauncher\debug\plUruLauncher.exe
.\plPythonPack\Debug\plPythonPack.exe

Now, time to delve into the SERVER. The next portion is Zrax's heavily edited README for his Dirtsand server. It is assumed that you have Ubuntu 10.10 installed, and that you wrote down the superuser password somewhere safe!

Phoenix says: "Yip, I had to reinstall Ubuntu once because I lost the root accounts password, so yes, it can happen to you too!"

On Ubuntu - DirtSand Howto

The original README is here...

• https://github.com/H-uru/dirtsand/blob/master/README

For ease of implementation we will be assuming that the server will be running on Ubuntu 10.10 (Maverick) Desktop version. This has proven to be the less painful way to delve into SERVER setup.

CAUTION: If you simply copy and paste Linux commands from this wiki, sometimes the terminal will execute the command automatically without you pressing enter. For safety, copy the command to a text editor, change the command as needed, then copy to the terminal window. Otherwise you may have to uninstall everything and start again!

Building the code

Prerequisites or dependancies are libraries or programs that another program needs to run or to compile properly. Without them you will get many errors. Do not proceed with this section of the wiki if you have not satisfied the list below.

Prerequisites:

• GCC 4.4+ (might work with other C++0x compliant compilers, but untested)
• Postgres (libpq) -- the actual database server can reside anywhere
• OpenSSL
• libreadline
• zlib
• git (to get the sources)

The default install of Ubuntu does not install all the items above. To install the packages that are lacking, open a terminal window, type the command below and follow the prompts before moving on to the next item in the list. This should satisfy the list of prerequisites above.

For those of you with Ubuntu 10.10, this is the easy part:

$ sudo apt-get install postgresql postgresql-contrib-8.4 libpq-dev apache2 cmake cmake-qt-gui g++ libreadline-dev

1) Now, create a user and path for the DIRTSAND server to run from.

$ sudo useradd dirtsand -d /opt/dirtsand -m -p <password> -s /bin/bash

Remember to drop the brackets <> when you create a password and watch your case (UPPERCASE or lowercase)!

2) Once done, unzip your copy of the "dirtsand" source code in your HOME directory.

$ cd dirtsand

3) Compile it for your system...

$ mkdir build && cd build
$ cmake -DCMAKE_INSTALL_PREFIX=/opt/dirtsand ..
$ make && sudo make install
$ cd ..

If you run into any errors about finding libraries or headers, make sure you have the *development* versions of all of the required libraries, and that they are in your path. If you really get stuck, you can also use the "cmake-gui" to help 'cmake' locate the missing paths and files.

$ cmake-gui

You have it already when you ran apt-get command earlier on.

Setting up a server - WITH a superuser account

You will need a working PostgreSQL server which DIRTSAND can use to store its data. Although you don't need superuser access to the postgres server, you will need to have a database which you can add schemas, tables, etc into.

For the default installation, the commands below will create a 'dirtsand' database and set its ownership to a 'dirtsand' database user, which can directly map the system 'dirtsand' user created in the "Building the code" instructions above. For better security, it is recommended to use a password (as shown in the steps below), which can be configured in the server settings as described in the "configure dirtsand" step.

1. Set up the postgres user

Now run the following commands and remember to drop the brackets <> around the password.

$ sudo -u postgres psql -d template1
template1=# CREATE USER dirtsand WITH PASSWORD '<password>';
template1=# CREATE DATABASE dirtsand WITH TEMPLATE = template0 ENCODING = 'UTF8';
template1=# ALTER DATABASE dirtsand OWNER TO dirtsand;
template1=# \q

2. Install the UUID functionality

This may be provided by your OS distribution. In Ubuntu, simply install the postgresql-contrib-8.4 package (Check the apt-get list at the start of the SERVER SECTION) to provide the necessary libraries and installation scripts. If your distribution does not provide a contrib or uuid-ossp bundle, you can get it and build it yourself from the sources provided at: http://www.ossp.org/pkg/lib/uuid/

Once you have the ossp, you can add it to the dirtsand database by running the command:

$ sudo -u postgres psql -d dirtsand < /usr/share/postgresql/8.4/contrib/uuid-ossp.sql

3. Set up the dirtsand database

$ sudo -u postgres psql -d dirtsand < db/dbinit.sql
$ sudo -u postgres psql -d dirtsand < db/functions.sql

NOTE: If you see the message 'ERROR: language "plpgsql" does not exist' while importing functions.sql, you will need to create the plpgsql language in the database:

$ sudo -u postgres psql -d dirtsand
dirtsand=# CREATE LANGUAGE plpgsql;
dirtsand=# \q

After this, you should re-import the functions.sql file to properly create the necessary dirtsand functions by running...

$ sudo -u postgres psql -d dirtsand < db/functions.sql

If there were no other errors, your database should be ready for DIRTSAND

4. Configure dirtsand

A sample dirtsand.ini has been provided in the root of the dirtsand sources. You will need to copy this to your install directory and then edit the fields you need for the server. Specifically, you will need to adjust the server addresses and the RC4 keys. If you have dirtsand installed to somewhere other than /opt/dirtsand, you will also need to point the configuration to the right paths too.

$ sudo cp dirtsand.sample.ini /opt/dirtsand/dirtsand.ini
$ sudo chown dirtsand /opt/dirtsand/dirtsand.ini
$ su - dirtsand
$ <your-favorite-editor> dirtsand.ini

If you use gedit as <your-favorite-editor> you may find it won't run in the dirtsand account, so you will need to exit the dirtsand user, use sudo then log back in to the 'dirtsand' user.

To generate the RC4 keys, you can simply run the keygen command from within the dirtsand interactive console:

$ bin/dirtsand
ds-902> keygen new       (This one will take a little while)
ds-902> quit

Any errors that dirtsand spits out about config files and postgres passwords can be ignored, since you haven't provided a configuration file yet.

You should now have a bunch of keys output on your terminal.

In your terminal, go to Edit, Select All, and Copy. Then paste the output to a file and save it as "Dirtsand.keys".

NOTE: You will need to copy this file to your Windows box later as well as the "dirtsand.ini" file.

The first block (labeled Server keys) in "Dirtsand.keys" is the set you should paste into your "dirtsand.ini" in the directory /opt/dirtsand/. Replace the dummy lines (with the '...' values) with the Server section in your "Dirtsand.keys" file.

FOR LATER on Windows: The second set of keys will be placed directly in the client's "server.ini" file.

NOTE: This requires either the H-uru fork of CWE or PlasmaClient.

Caution -- the vanilla MOUL client cannot load keys from a file, and you will have to enter the keys as byte arrays directly into the executable. This will not be covered here.

Later we will explain how to add the Client keys on the Windows box.

Now before we move on, you need to unzip the 'moulscripts' Python27 branch to your HOME folder.

When you have unzipped the "Python27" branch of moulscripts, you will see 3 folders in "Python27" folder:

./dirtsand/dat
./dirtsand/Python
./dirtsand/SDL

One of these needs to be renamed to fall in line with the naming convention in the dirtsand.ini file.

Rename "./Python27/dat" to "./Python27/ages" (This is to save you a couple of head scratches later on) like so:

$ mv dat ages

The files inside these folders are all the age, Python and SDL information dirtsand needs for the ages in UN-encrypted format.

Leave them unencrypted for now because you will see later you need two copies of these files in unencrypted and encrypted format! More on that later...

CAUTIONARY NOTE: Since Plasma, dirtsand and moul-scripts are in HEAVY alpha, you might find things don't compile or throw out lots of errors. First check that you are compiling on a 32 bit WinXp system. Win7 64 bit is a little rebellious when it comes to 32 bit compatibility.

On Windows - Authdata, Manifests and Data

5) Now, move over to your Windows box.

Firstly, you should copy the Dirtsand.keys and dirtsand.ini file over to your Windows box now, and place them in their own folder for easy reference.

For data serving to work you will need to create some server-side data such as manifests, encrypted python.pak and the content (data) of MOUL, but make special note that although this is created on Windows, you will copy the newly created files over to your UBUNTU box later!

NOTE: Make sure to use the Win32 version of libhsPlasma to avoid any potential errors creeping in.

The two tools you will use is plPythonPack.exe that was compiled earlier on and PlasmaCrypt.exe in your "Plasma_Win32" (libhsplasma) download.

CAUTION: I tried to originally compile the client on Win7 64bit machine. There were some major errors I could not solve. As soon as I compiled a fresh build on a WinXP 32 bit system, everything fell into place.

Okay lets get going... What we found is this; dirtsand needs two versions of the "ages", "Python" and "SDL" files. One version is UN-encrypted and the other is encrypted, but each goes in a specific place.

Note: When we refer to encrypted Path, we mean; the files inside the folders are encrypted, NOT THE FOLDER ITSELF!

Unencrypted Paths: <For dirtsands first launch>

/opt/dirtsand/ages/ <holds un-encrypted file versions of moulscripts>
/opt/dirtsand/Python/ <holds un-encrypted file versions of moulscripts>
/opt/dirtsand/SDL/ <holds un-encrypted file versions of moulscripts>

Encrypted Paths:

/opt/dirtsand/authdata/ages/ <holds encrypted file versions of moulscripts>
/opt/dirtsand/authdata/Python/ <holds encrypted file versions of moulscripts>
/opt/dirtsand/authdata/SDL/ <holds encrypted file versions of moulscripts>

dirtsand.ini searches for these locations by default:

File.Root = /opt/dirtsand/data
Auth.Root = /opt/dirtsand/authdata
Sdl.Path = /opt/dirtsand/SDL
Age.Path = /opt/dirtsand/ages

Now the one conflict here is that 'dat' and 'data' can be confusing

1. 'dat' must be renamed to 'ages' to align to 'Age.Path' in dirtsand.ini (See above)
2. 'data' is the copy of MOULa content in the "dat" folder of your fresh MOUL install off your windows box.

For dataserving, the most important server-side data for the client to access are the "authdata" server-provided files -- specifically, the ENcrypted "ages" and "SDL" and "python.pak" files.

SIDE NOTE: External Plasma clients require these files in order to function, so you will need to provide them unless you are planning on making your server only work with Internal client builds.

To create the files, do the following:

NOTE: If you want to setup a PATH variable and shortcut this process, feel free to do so if you know how.

1. Make a copy of the "Python27" folder, and then do 2 things; rename the copied folder to "authdata" AND move it INTO the "Python27" folder. Now be careful with the next two steps!!
2. copy plPythonPack.exe, msvcr100d.dll and python27_d.dll into the "authdata" folder.
3. Launch "cmd" and browse to the "moulscripts\Python27\authdata" folder
4. in a windows terminal type...

plPythonPack.exe Python

...and press enter. The command should return without errors. If there any errors you may have used a 64 bit system to compile "plPythonPack.exe".

Your windows folder structure should look like this;

\Python27\ages
\Python27\Python
\Python27\SDL
\Python27\authdata\ages
\Python27\authdata\Python
\Python27\authdata\SDL

You should notice 1 new file in the "\Python27\authdata\Python" folder named python.pak.

• \Python27\authdata\Python\python.pak

Although we have just packed the Python folder into "python.pak" it is still not encrypted.

To encrypt this file set up a path variable to libhPlasma tools on Windows.

Once you have set up the PATH variable, reboot for the changes to take effect.

Now you can simply use "cmd" to navigate to the authdata folder.

Now, the files that needs to be encrypted is the python.pak file you created earlier and all the "SDL" and "ages" files in the "\Python27\authdata\" folder.

Navigate to the \Python27\authdata\Python\ folder and type (drop the inverted brackets in the command below):

prompt:> PlasmaCrypt droid -key {paste the key.droid value from ".\CWE Sandbox\INI Files\dirtsand.ini" here} python.pak

It should look like;

prompt:> PlasmaCrypt droid -key 86756......283746 python.pak

This 'key.droid' must be 32 digits long.

Next you need to encrypt the contents of the "ages" and "SDL" folders.

NOTE: Unfortunately there is no recursive support in plasmacrypt, so either do each file in each folder one by one or use the technique below. The technique below is a little time consuming, but you will only have to do it once.

We are going to create a spreadsheet for each 'ages' and 'SDL' folder that will be used to create to batch files.

Your final list of commands in each batch file should look like:

PlasmaCrypt droid -key <key here> Ahnonay.age
etc
etc
etc

You can build a list of files using the "dir -b > sdl-enc.csv" and "dir -b > ages-enc.csv" command piped to a file, so that you can edit it in scalc or excel;

1. Add droid key to spreadsheet
2. save as "csv" with space seperator
3. edit with scite or notepad, and remove all spaces and inverted commas
4. save the files with a "bat" extension into their respective folders
5. use the terminal and browse to each folder and execute the batch file.
6. If you did it right, when you open any one of the files in scite, it will look like gibberish with the starting line, "NotTheDroids"

Now you are ready to create the manifest files.

MANIFEST FILES - Creating the manifest on Windows (will be copied to Ubuntu later):

The server tells the client which files are available. For this you will also need to provide the following files in the "authdata" folder (these are called manifests);

• python_pak.list
• SDL_sdl.list
• AGES_ages.list

...which are simple manifest files of the format:

 Path\filename.ext,size-in-bytes

So, inside the python_pak.list file, you might have:

 Python\python.pak,123456

And inside the SDL_sdl.list file you would have;

 SDL\ahnonay.sdl,27311
 SDL\ahnonaycathedral.sdl,3524
 SDL\etc,3453
 SDL\etc,476585
 SDL\etc,345
 SDL\etc,23

Note the use of a backslash here, since this path is provided directly to the client (which assumes a Windows path).

To create these files you will need the command prompt, a text editor and a spreadsheet program.

Open a text editor and save a blank file as Python_pak.list

In the command prompt, navigate to the "\Python27\authdata\Python\" folder and type:

prompt:> dir

The entry for python.pak should have a file size in bytes alongside it to the left.

Type out the file size in bytes into the text editor (without spaces):

  Python\python.pak,***

becomes (the value to the right is only an example)

  Python\python.pak,510544

save the file as "Python_pak.list", and you are done with your first manifest list.

Next... change to the SDL folder...

prompt:> dir SDL > sdl.txt

This will create a file called sdl.txt in the root of "\Python27\authdata\SDL\" with all the file sizes in it. Now it is easy to load this into a spreadsheet program and do a clean up. The list should look like;

 SDL\ahnonay.sdl,27311
 SDL\ahnonaycathedral.sdl,3524
 SDL\etc,3453
 SDL\etc,476585
 SDL\etc,345
 SDL\etc,23

In scalc or excel a typical row would look like: (after all your fixing!)

Now, save it as csv (with comma as the separator), and open it in a text editor, and check it by hand to make sure it is right as shown above.

Once you are happy with it, change the name of the file to SDL_sdl.list

That is manifest file number 2 done!

Now do the same for "ages". The result should be AGES_ages.list.

NOTE: Remember, don't confuse "data" with "dat" in moulscripts.

 "dat" should rightly be renamed to "ages"

 "data" is a renamed copy of the "dat" folder in your fresh MOUL install that dirtsand will use to "serve" data (levels) to the client.

Now, assuming you have prepared a fresh MOUL installation into your Games folder, browse to it and copy the "dat" content inside the MOUL folder to a new folder called "data" and place this folder (as big as it is) in your "Python27" folder.

So now, in "\Python27\" you should have:

\Python27\ages\
\Python27\data\
\Python27\SDL\
\Python27\Python\
\Python27\authdata\ages\
\Python27\authdata\SDL\
\Python27\authdata\Python\
\Python27\authdata\Python\python.pak
\Python27\authdata\AGES_ages.list
\Python27\authdata\SDL_sdl.list
\Python27\authdata\python_pak.list

Now before moving on, you will need to copy the encrypted contents of the '\authdata\ages\' folder into the 'data' folder... The reason is that the original content from MOUL is encrypted for Cyans droid key and not your Sandbox Shard, so you need to copy and overwrite all the *.age and *.fni files in \Python27\data\ with your newly encrypted copies from \Python27\authdata\ages.

Now... copy the entire Python27 folder to the HOME folder on your Ubuntu box.

On Ubuntu - Bringing in Authdata, Manifests and Data into Dirtsand

6) "Authdata" files

For dirtsand to provide the game files to the client, we need to do four things:

• 1) set up unecrypted versions of the ages, python and SDL folders in the root of dirtsand, and
• 2) set up a directory for "authdata" for the encrypted ages, python and SDL folders in the dirtsand folder.
• 3) bring in the data with encrypted age and fni files and process those files with dsData.sh
• 4) place the manifests in the 'authdata' folder.

Here are some of the spreadsheets and scripts created for this purpose; Use at your own risk and double check everything!

http://www.falsebaypost.co.za/cwe-sandbox/Sandbox Scripts.zip

Step 1: Dirtsand "root" folder

The reason for the first step above is that when dirtsand is launched for the first time, dirtsand needs to read the un-encrypted "ages", "Python" and "SDL" files to configure properly, so unencrypted versions are placed in the root of dirtsand...

/opt/dirtsand/

In this folder you will have unencrypted copies of "ages", "Python" and "SDL" so that dirtsand can configure properly when run for the first time.

/opt/dirtsand/ages/<unecrypted age and fni files>
/opt/dirtsand/SDL/<unecrypted SDL files>
/opt/dirtsand/Python/<unecrypted Python files>
and
/opt/dirtsand/data/<converted data files to send to client>

Step 2: Dirtsand "authdata" folder

Now to before we move on, remember we will be creating the "authdata" folder to point to...

 /opt/dirtsand/authdata)

and then putting the "ages", "Python" and "SDL" folders and their ENCRYPTED files in the respective subdirectories within the "authdata" folder.

The folders:

 /opt/dirtsand/authdata/ages
 /opt/dirtsand/authdata/Python
 /opt/dirtsand/authdata/SDL

The files:

 /opt/dirtsand/authdata/<manifest>.list
 /opt/dirtsand/authdata/ages/*.age
 /opt/dirtsand/authdata/ages/*.fni
 /opt/dirtsand/authdata/Python/python.pak
 /opt/dirtsand/authdata/SDL/*.sdl

NOTE: Remember that the manifest *.list files are NOT encrypted and go directly in your "authdata" folder.

Now remembering the structure above, plug in your memory stick in Ubuntu, and then in Nautilus (or a terminal) copy the Python27 folder to your HOME directory.

Now before you can add the files to dirtsand you must change their ownership otherwise you'll be back at this step later! ; )

Open a terminal (it opens in your HOME directory by default) and type:

$ sudo chown -R dirtsand Python27

The -R flag will make sure "chown" changes the ownership of all files... even inside folders.

Now, we can go to our terminal window, and type:

$ su - dirtsand

If you type "ls" you will see that you are in the root of the "dirtsand" folder

/opt/dirtsand/

Now using a terminal you can copy the files over;

$ cp -R /home/<username>/Python27/*.* /opt/dirtsand/.

3) Data Server

To use the data server with its services you will need to run the 'dsData.sh' script on the content of the data folder in;

/opt/dirtsand/data/

NOTE: There is no recursive functionity in dsData.sh, so you can do this one file at a time or follow the steps below and use the spreadsheet method.

NOTE: At a later date you may choose to create the ThinExternal.mfs and External.mfs files, which describe the files necessary for the patcher and client (respectively) to run. Don't worry about this now.

You will need to create 'age', and 'prp' manifests of the form "<agename>.mfs" which will be requested by the client when it attempts to link to an age.

NOTE: Dont process the 'fni' files with dsData.sh. Just leave them as they are.

The format of the manifest files is as follows:

remote_filename.ext,local_filename.ext.gz,decompressed_hash,compressed_hash,decompressed_size,compressed_size,flags

The helper script provided in bin/dsData.sh will gzip a file and generate a manifest line with the correct hashes and sizes for you automatically. Take care that the remote path expects to use a Windows path/filename, so it should use backslashes instead of forward ones, whereas the local filename should use Unix slashes.

The best way to start this process of converting the data folder to the correct format is to run the command;

$ ls -b /opt/dirtsand/data >> data.list

This will create a file with a listing of the filenames from the 'data' folder.

Now load it into 'scalc' or other spreadsheet program and create a row for each file as follows;

/opt/dirtsand/bin/dsData.sh AhnonayCathedral.age >> AhnonayCathedral.mfs;
/opt/dirtsand/bin/dsData.sh AhnonayCathedral_District_BuiltIn.prp >> AhnonayCathedral_District_BuiltIn.mfs;
/opt/dirtsand/bin/dsData.sh AhnonayCathedral_District_Textures.prp >> AhnonayCathedral_District_Textures.mfs;
etc;
etc;
etc;

Do this for EVERY age and prp file. Be careful to include the semi-colon at the end of each line.

Save this file as a 'csv' and open it with a text editor. The remove all the fluff that does not look like the format description above.

Now save it with the extension .sh and run it from inside the 'data' folder.

When it is done you should have a ".gz" and a ".mfs" version for each file. You wont see the original files anymore because they have been packed into the ".gz" archive.

Now comes the tricky part... In each .mfs file you need to append the tag "dat\" at the beginning of the line. This corresponds to the 'dat' folder in your MOUL install on windows. If this is not included, the client will write the updated data into the root of your MOUL.

To append 'dat\', you can either do this by hand... and take a week to do it, or build a spreadsheet to help you create a script, which means an hour or so tops out of your time.

Start by creating a small file called "stub" in the 'data' folder. Inside it put the line "dat\" and save it.

The spreadsheet layout should be as follows; (Don't forget the semi colon at the end of each line.)

#! bin/bash
paste stub AhnonayCathedral.mfs | sed 's/\t//' >> AhnonayCathedral.mfs1;
paste stub AhnonayCathedral_District_BuiltIn.mfs | sed 's/\t//' >> AhnonayCathedral_District_BuiltIn.mfs1;
paste stub AhnonayCathedral_District_LinkRoom.mfs | sed 's/\t//' >> AhnonayCathedral_District_LinkRoom.mfs1;
paste stub AhnonayCathedral_District_Textures.mfs | sed 's/\t//' >> AhnonayCathedral_District_Textures.mfs1;
etc;
etc;
etc;

Now save it as 'csv' and open it in a text editor to clean it up so it looks like above.

What this will do is join the line "dat\" to the line inside each ".mfs" file and save it as a new ".mfs1" file.

Then to get the filename corrected, once you check that the ".mfs1" file contents is correct by using your trusty text editor, delete all the ".mfs" files (or backup them up somewhere else) and run the next command based on another spreadsheet;

#! bin/bash
mv AhnonayCathedral_District_BuiltIn.mfs1 AhnonayCathedral_District_BuiltIn.mfs;
mv AhnonayCathedral_District_LinkRoom.mfs1 AhnonayCathedral_District_LinkRoom.mfs;
mv AhnonayCathedral_District_Textures.mfs1 AhnonayCathedral_District_Textures.mfs;
etc;
etc;
etc;

Now you should have a ton of ".gz" and ".mfs" files in the data folder in dirtsand.

Remember that if you set up another shard, you can simply reuse your new scripts that you built from spreadsheets, so this big task is now finally done and you wont need to play with spreadsheets beyond this point.

On Windows - Building a new MOUL install

8) Setting up your Windows Client:

If you want to setup your Windows client from scratch with your compiled versions plClient and plUruLauncher and your new server.ini file, follow these steps:

Install MOULInstaller887.exe (There is a later version MoulInstaller887.exe)

Watch out when the installer finishes... it tries to connect to Cyan, so make sure you got your network cable unplugged temporarily. You don't want their updates for your shard! The reason is, they are based on python 2.3, and your shard only runs python 2.7.

Then in the root of your fresh Sandbox installation do this:

• Backup your UruLauncher.exe
• Backup your UruExplorer.exe
• Copy your compiled plClient.exe from .\CWE Sandbox\plEngine\ to the root of your Sandbox install and rename it to UruExplorer.exe
• Copy your compiled plUruLauncher.exe .\CWE Sandbox\plEngine\ to the root of your Sandbox install and rename it to UruLauncher.exe
• Copy wrap_oal.dll
• Copy NxCharacter.dll
• Copy NxCooking.dll
• Copy NxExtensions.dll
• Copy OpenAL32.dll
• Copy python27.dll
• Copy python27_d.dll
• Copy resource.dat
• Edit tos.txt with the GNU GPL license

Then to setup the server.ini file...

• Copy the Zrax's ini file at http://moul.zrax.net/zraxnet.ini and save it to your Sandbox.
• Rename zraxnet.ini to mynet.ini
• Open a Windows Command Prompt, change to that directory and run the following command:

Prompt:> PlasmaCrypt decrypt mynet.ini

Once the command is finished, open the file with a text editor (Use one that keeps format in place - Like Scite or Scintilla, just easier than notepad)

Open your Dirtsand.keys file and paste your 6 Client auth keys that were generated from dirtsand over the 6 Client auth keys in the mynet.ini file. If you open the mynet.ini file on your windows box and see mumbo jumbo, you did not decrypt your file correctly, so try the decrypt command again.

Now the last entries in the mynet.ini file are:

   Server.Gate.Host "xxx.xxx.xx.xxx" (IP address assigned to Ubuntu -

       - to find this go to your ubuntu terminal and type ifconfig, and use the address after "inet addr:")

   Server.Auth.Host "xxx.xxx.xx.xxx" (same as above)

   Server.DispName "The journey continues..." (whatever catch phrase kicks your hair back)

   Server.Status "<same as above>/shardstatus.php" (contains a simple PHP print command,

       - eg: 'hello world' or 'server is up' - Needs explanation on setting up simple Apache -   Anyone?

   Server.Signup "<same as above>/newacct.php" (no idea how to create the account creation file...

       - and how it ties back to the dirtsand command "addacct <username> <password>" ) - Anyone?

Once done you will need to encrypt the mynet.ini file with the 'xtea' parameter in PlasmaCrypt. Just run the command:

Prompt:> PlasmaCrypt xtea mynet.ini

The command returns a success if all went well, copy mynet.ini to your Sandbox install on windows and rename it to 'server.ini'

On Ubuntu - Running the Sandbox Server, pgAdmin3 and Wireshark

9) Run the server:

Assuming everything else went smoothly, you should now be able to start your server and connect to it!

To start the server... You will use this command every time you want to start dirtsand after a reboot.

$ bin/dirtsand dirtsand.ini

You'll have to create an account first, which can be done from the console:

$ ds-902> addacct <username> <password>

NOTE: The first time you run this can be quite odd since no vault is yet created. It seems to error out. When it happens just press enter and you should get to the dirtsand prompt.

If you need to verify that an account has been made, or you want to check that your server is listening for your windows client, you can follow the pgadmin and wireshark steps here...

On Windows - Logging into the Sandbox Server

10) Run the Client: On Windows...

Launch plClient and enter the username and password you created in the step above.

After the license screen CWE should load and you will be taken to menu to create an avatar. You know the rest.

Happy Exploring!

Additional Information

Git/download links:

• http://github.com/H-uru/dirtsand
• http://github.com/H-uru/Plasma
• https://github.com/H-uru/moul-scripts
• https://github.com/H-uru/libhsplasma
• http://guildofwriters.com/tools/devlibs.zip

SETTING UP pgAdmin3:

To check if you have created a user successfully, you will need to download pgadmin3...

$ sudo apt-get install pgadmin3

This will help you access postgres in a friendly gui environment. Setting up pgadmin3 is quite simple once it is installed. Click on "Applications" , "Programming" , "pgAdmin3"

The gui will launch. Once open click on the icon thats looks like an electrical plug to create a new server connection.

The fields should be exactly as below unless you changed them when you set up postgres a number of steps back.

  Name: dirtsand
  Host: 127.0.0.1
  Port: 5432 - (leave this as it is)
  SSL: blank - (Not important right now... keep it simple)
  Maintenance DB: postgres
  Username: dirtsand
  Password: <whatever you chose for your password>

Leave the rest of the window as it is. You will notice the windows has no "OK" button. Just after you entered your password, please "Enter" and the connection will be made. A dialog will pop up about "Saving passwords", read it and click "OK".

If it fails, you entered something wrong, so try again by clicking on the plug icon.

You know you have succeeded to login to the database when in the left hand pane you see;

"Server Groups", below that "Servers", and below that "dirtsand"

To check the new player you added, click on "dirtsand", then the + sign to the left of it to expand it. Then "Databases" , "dirtsand" , "Schemas" , "auth" , "Tables" , "Accounts"

Now, look for the little blue icon in the toolbar that looks like a spreadsheet... and click it.

You should be looking at your first account that you created with the command;

 ds-902> addacct <username> <password>

If you dont see the exact username you created, you will have to backtrack your steps in this wiki, and try to figure out where you went wrong.

If all looks well, save your connection... "File" , "Save Definition" , then give it a cool name like "Sandbox Shard" and click "Save" otherwise when you next launch pgAdmin3 you will have to manually reconnect again.

SETTING UP Wireshark:

This tool is very useful to check if your client is connecting to your Ubuntu shard. To install it run...

$ sudo apt-get install wireshark

It is best to do a reboot, and when back in Ubuntu DON'T run it from the "Applications" menu...

Open a terminal, and type...

$ sudo wireshark

NOTE: If you run it from the App menu you won't find your ethernet controller.

Now, once wireshark has launched you will get a warning. Acknowledge it, and click "OK". Please don't do anything rash while using wireshark. Just follow these instructions.

Once you click "OK" you should be taken to the wireshark main interface. Look under Capture on the main page, and identify your ethernet controller. Generally it is called "eth0" but you may have some other configuration.

If you click on "eth0" wireshark will start to capture all traffic over that interface immediately. This is a good sign that wireshark and your ethernet controller are both working!

Stop the live capture by clicking on the icon with the little red cross.

Click on "Capture" in the top menu, and click "Options"

Now, be careful... the only field you want to fill out is the one to the right of "Capture Filter".

You will need to jump to your Windows box quickly and in a command terminal run

  prompt:> ipconfig

Look through the list and look for the IPv4 address for your computer... Something like this;

Ethernet adapter Local Area Connection:

 Connection-specific DNS Suffix  . :
 Link-local IPv6 Address . . . . . : ****::****:****:****:***%**
 IPv4 Address. . . . . . . . . . . : 124.122.23.11  << Your IP address will be here!
 Subnet Mask . . . . . . . . . . . : 255.255.255.0
 Default Gateway . . . . . . . . . : 235.211.11.1

Now back on Ubuntu fill in that field just mentioned, type;

host 124.122.23.11 (replace with whatever your IP is - the word "host" is crucial!)

Do not change any settings beside this, and simply click "START", then "Continue without Saving"

What this will do is ignore all other internet traffic except traffic from your Windows client. This makes reading wireshark much easier than having to sift through a ton of traffic!

The screen should be blank, so first we need to test the connection. First, in a new terminal on Ubuntu, run...

$ /sbin/ifconfig

Look for "inet addr" the IP address under "eth0" of your Sandbox and write it down. Take this to your Windows box.

On your Windows box, your command prompt should still be open after you ran ipconfig. Now run a ping command...

prompt:> ping ***.***.**.** (type out the full IP address)

Now brace yourself, hehe! As you hit "enter" keep a beady eye on your Ubuntu wireshark screen. You should get about 10 lines of connection reports. This is great... your client is "seeing" your Server!

Now in wireshark on Ubuntu click "Capture" , "Restart" to clear the screen.

Here is where you now hold your breath, because you are about to see if your recently compiled plClient.exe (and all the work you have done) can establish a connection with your shard.