Setting Up A Tiny Sandbox Shard

How to create a TINY SANDBOX SHARD without Dataserver

For Non-Programmers

License: GNU GPL

by Phoenix Rising version 0.02 August 2011

---

Shorah Explorers

====AIM:==== The aim of this wiki is to teach a writer/builder/developer how to build a tiny sandbox shard without dataserver to test CWE compatible ages. This is the smallest possible shard you can build.

Once you have cut your teeth on this wiki page, feel free to try out the Sandbox Shard with Dataserving. This one by far is the easiest to do, so try this one first.

The aim of this document is to detail as closely as possible the exact steps to create your own Tiny Sandbox Shard (without dataserver) using GoW's CWE Client and Zrax's Dirtsand Server. This wiki page is specifically for writers and age builders who want to test MOULa compatible ages without Dataserving capability.

We have permission from Zrax to use his README in this wiki but with a much expanded and detailed version due to a few undocumented features. We have chosen to use GoW's CWE Client since it has been heavily updated so that it may be compiled easily and with a much more recent version of DirectX SDK and other tweaks such as reduction of lag. This simply means that it has better support and hopefully better gameplay on your fancy new computer!

====IMPORTANT NOTE:==== This document gets highly technical, and as such is not recommended for newbies. This is almost complete so if you want to add to this wiki, please get persmission from the author first! We want this to appeal to every Non-Programmer young and old and have a really cool wiki to show for it. If you have helpful suggestions, drop the author a post. A lot of hard work has gone into documenting this procedure for the sake of all the Uru fans out there. At the end of the day all this effort is due to teamwork, since no-one can do such a mammoth task alone!

====HARDWARE PREREQUISITES:==== At least two computers are needed (one of which must have an able graphics card).

As a benchmark we will be building the Tiny Sandbox based on:

• CLIENT MACHINE: A Dell Inspiron running Windows 7 64 bit, with a 1Gb Ati Graphics card, and
• SERVER MACHINE: An Asus Eeepc mini laptop running Ubuntu 10.10 (Maverick).

The specs are not crucial, except for the graphics card on the Windows Machine and the Flavour of Linux on the SERVER. We have chosen Ubuntu 10.10 because although we still have to configure stuff, we dont like configuration nightmares!

A simple test to see if your Windows CLIENT can handle all of CWE's graphics requirements is to ramp up all your Graphics settings to maximum when you run the launcher for TPOTS and see if your computer runs smoothly or has screen jitter/stutter. If your machine can handle these high settings you will be less restricted in your age building later on.

====NETWORK CONNECTION:==== This wiki will attempt to document one of the two approaches to network connectivity.

1. SANDBOX MODE - SERVER and CLIENT on a local hub using DHCP or fixed IP.
2. SHARD MODE - SERVER on dynamic DNS or fixed IP, and CLIENT accessing through the internet.

SHARD MODE will be covered in another wiki at a later date.

Both configurations have their merits. The first provides the "offline" sandbox mode for creating your own ages and testing the content, even though you will still be connecting via a network of some kind. The second makes your shard accessible to other players over the internet. This you will use when you are ready to go live!

If you want to start building ages, we do not recommend following this wiki beyond the following few lines. There are many tutorials to help you build your first age.

---

Assumptions and Caveats to follow this Wiki

• You will be compiling the CWE client on a WinXp 32 Bit machine.
• You know how to use Windows command line
• You know how to use a Linux command line
• You know how to compile C++ software on Windows
• You know how to compile C++ software on Linux
• You are fluent using both systems and understand the subtleties of each, such as: case sensitivity, backslash versus forward slash, and what a terminal is.
• You know how to use scalc or excel
• You know what a batch file is
• YOu know what a script is

If you fail to do any of the above, we really suggest that you do not attempt this.

The major caveat is that will have have to create one spreadsheet that is used as a template to create a batch file. As tedious as it may seem this is due to a few issues;

• The official plasma tools does not support recursion.
• Your newly compiled plPythonPack does not support recursion.
• If you had to encrypt one file at a time you would be here for two weeks. Just take a look at how many files are in the 'SDL', 'age' and 'dat' folders!
• The scripts give you a way to update bulk content packages aimed at the content YOU want on YOUR shard.
• Once the scripts are created, you wont have to create them again, just remember to save them somewhere safe.

Ok, let's begin:

As this wiki unfolds you will notice that we will have to do work on both the SERVER and the CLIENT at any one time, so be prepared for it before it happens. We will refer to the client as Windows, and the server as Ubuntu for ease of reference.

Github and the branches

There are 3 main projects that we will pull from github.

1. https://github.com/H-uru/Plasma
2. https://github.com/H-uru/dirtsand
3. https://github.com/H-uru/moul-scripts/tree/python27

An easy way to get the code for each is to click the "Downloads" button to the far right of "branches" tab.

You will also need to download the windows version of plasma tools to help you encrypt the various files later.

• http://libhsplasma.googlecode.com/files/Plasma_Win32_r848.zip

(Note, we won't be covering how to build a Cyan compatible engine, since Cyans MOUL still runs on Python 2.3 and CWE runs on Python 2.7)

On Windows - Let's compile CWE

Our starting point is to build the CWE client based on the "master" branch on github since it is a relatively simple process and will help you get your feet wet.

To build the client you can follow the GoW tutorial and video clip below:

• Tut: http://www.guildofwriters.com/wiki/Development:CyanWorlds.com_Engine
• Clip: http://www.youtube.com/watch?v=HZbMf5W67UI

An Undocumented item in the above tut: Just remember that you want to compile a 'Release' build. The default tut above compiles a 'debug' build. When you run a debug build it throws out errors with numerous dailog boxes, so it is preferable to build the 'Release' build for uninterrupted gameplay.

Here is a brief comparison of the two builds and their dependancies; (You can find these dll's in the 'cwe-prefix' folder)

• Release version needs - python27.dll
• Debug version needs - python27_d.dll and ms****.dll's

In Visual Studio simply go to the 'Build' menu, 'Configuration Manager', 'Active Solution Configuration' and select 'Release' and save the project.

Once you have compiled it, you will have 3 files in;

.\Plasma\build\Sources\Plasma\Apps\

They are;

.\plClient\release\plClient.exe
.\plUruLauncher\release\plUruLauncher.exe
.\plPythonPack\release\plPythonPack.exe

We will come back to this later.

Now, time to delve into the SERVER. This is based on Zrax's heavily edited README for his Dirtsand server. It is assumed that you have Ubuntu 10.10 installed, and that you wrote down the superuser password somewhere safe!

On Ubuntu - DirtSand Howto

The original README is here...

• https://github.com/H-uru/dirtsand/blob/master/README

For ease of implementation we will be assuming that the server will be running on Ubuntu 10.10 (Maverick) Desktop version. This has proven to be the less painful way to delve into SERVER setup. There are some undocumented issues that are clarified in this wiki page.

CAUTION: As you progress with this section, be aware that if you simply copy and paste Linux commands from this wiki, sometimes the terminal will execute the command automatically without you pressing enter. For safety, copy the command to a text editor, change the command as needed, then copy to the terminal window. Otherwise you may have to uninstall everything and start again!

Building the code

Prerequisites or dependancies are libraries or programs that another program needs to run or to compile properly. Without them you will get many errors. Do not proceed with this section of the wiki if you have not satisfied the list below.

Prerequisites:

• GCC 4.4+ (might work with other C++0x compliant compilers, but untested)
• Postgres (libpq) -- the actual database server can reside anywhere
• OpenSSL
• libreadline
• zlib

The default install of Ubuntu does not install all the items above. To install the packages that are lacking, open a terminal window, type the command below and follow the prompts before moving on to the next item in the list. This should satisfy the list of prerequisites above.

You may find that "libpq" will not install. This only happens on a fresh install of Ubuntu 10.10. Simply update the essential updates for Ubuntu, reboot, and try again.

For those of you with Ubuntu 10.10, this is the easy part:

$ sudo apt-get install postgresql postgresql-contrib-8.4 libpq-dev apache2 cmake cmake-qt-gui g++ libreadline-dev

1) Now, create a user and path for the DIRTSAND server to run from.

$ sudo useradd dirtsand -d /opt/dirtsand -m -p <password> -s /bin/bash

Remember to drop the brackets <> when you create a password and watch your case (UPPERCASE or lowercase)!

2) Once done, unzip your copy of the "dirtsand" source code in your HOME directory.

$ cd dirtsand

3) Compile it for your system...

$ mkdir build && cd build
$ cmake -DCMAKE_INSTALL_PREFIX=/opt/dirtsand ..
$ make && sudo make install
$ cd ..

If you run into any errors about finding libraries or headers, make sure you have the *development* versions of all of the required libraries, and that they are in your path. If you really get stuck, you can also use the "cmake-gui" to help 'cmake' locate the missing paths and files.

$ cmake-gui

You have it already when you ran apt-get command earlier on.

Setting up the Server - WITH a superuser account

You will need a working PostgreSQL server which DIRTSAND can use to store its data. Although you don't need superuser access to the postgres server, you will need to have a database which you can add schemas, tables, etc into.

For the default installation, the commands below will create a 'dirtsand' database and set its ownership to a 'dirtsand' database user, which can directly map the system 'dirtsand' user created in the "Building the code" instructions above. For better security, it is recommended to use a password (as shown in the steps below), which can be configured in the server settings as described in the "configure dirtsand" step.

1. Set up the postgres user

Now run the following commands and remember to drop the brackets <> around the password.

$ sudo -u postgres psql -d template1
template1=# CREATE USER dirtsand WITH PASSWORD '<password>';
template1=# CREATE DATABASE dirtsand WITH TEMPLATE = template0 ENCODING = 'UTF8';
template1=# ALTER DATABASE dirtsand OWNER TO dirtsand;
template1=# \q

2. Install the UUID functionality

This may be provided by your OS distribution. In Ubuntu, simply install the postgresql-contrib-8.4 package (Check the apt-get list at the start of the SERVER SECTION) to provide the necessary libraries and installation scripts. If your distribution does not provide a contrib or uuid-ossp bundle, you can get it and build it yourself from the sources provided at: http://www.ossp.org/pkg/lib/uuid/

Once you have the ossp, you can add it to the dirtsand database by running the command:

$ sudo -u postgres psql -d dirtsand < /usr/share/postgresql/8.4/contrib/uuid-ossp.sql

3. Set up the dirtsand database

$ sudo -u postgres psql -d dirtsand < db/dbinit.sql
$ sudo -u postgres psql -d dirtsand < db/functions.sql

NOTE: If you see the message 'ERROR: language "plpgsql" does not exist' while importing functions.sql, you will need to create the plpgsql language in the database:

$ sudo -u postgres psql -d dirtsand
dirtsand=# CREATE LANGUAGE plpgsql;
dirtsand=# \q

After this, you should re-import the functions.sql file to properly create the necessary dirtsand functions by running...

$ sudo -u postgres psql -d dirtsand < db/functions.sql

If there were no other errors, your database should be ready for DIRTSAND

4. Configure dirtsand

A sample dirtsand.ini has been provided in the root of the dirtsand sources. You will need to copy this to your install directory and then edit the fields you need for the server. Specifically, you will need to adjust the server addresses and the RC4 keys. If you have dirtsand installed to somewhere other than /opt/dirtsand, you will also need to point the configuration to the right paths too.

$ sudo cp dirtsand.sample.ini /opt/dirtsand/dirtsand.ini
$ sudo chown dirtsand /opt/dirtsand/dirtsand.ini
$ su - dirtsand
$ <your-favorite-editor> dirtsand.ini

If you use gedit as <your-favorite-editor> you may find it won't run in the dirtsand account, so you will need to exit the dirtsand user, use sudo then log back in to the 'dirtsand' user.

To generate the RC4 keys, you can simply run the keygen command from within the dirtsand interactive console:

$ bin/dirtsand
ds-902> keygen new       (This one will take a little while)
ds-902> quit

Any errors that dirtsand spits out about config files and postgres passwords can be ignored, since you haven't provided a configuration file yet.

You should now have a bunch of keys output on your terminal.

In your terminal, go to Edit, Select All, and Copy. Then paste the output to a file and save it as "Dirtsand.keys".

NOTE: You will need to copy this file to your Windows box later as well as the "dirtsand.ini" file.

The first block (labeled Server keys) in "Dirtsand.keys" is the set you should paste into your "dirtsand.ini" in the directory /opt/dirtsand/. Replace the dummy lines (with the '...' values) with the Server section in your "Dirtsand.keys" file.

FOR LATER on Windows: The second set of keys will be placed directly in the client's "server.ini" file.

NOTE: This requires either the H-uru fork of CWE or PlasmaClient.

Caution -- the vanilla MOUL client cannot load keys from a file, and you will have to enter the keys as byte arrays directly into the executable. This will not be covered here.

Later we will explain how to add the Client keys on the Windows box.

5. Setting up a data-less Server

To setup the dataless server, (instead of copying the MOUL content to dirtsand for dataserving) we need to create some zero byte files and place them where they belong. When the client starts up it will still request data from the server, and the zero byte files tell the client there is nothing to download.

First off update your dirtsand.ini file;

File.Root = /opt/dirtsand/tiny/data
Auth.Root = /opt/dirtsand/tiny/authdata

To create the empty files: (You should still be logged in as dirtsand)

$ cd
$ cd tiny
$ mkdir data authdata
$ touch authdata/Python_pak.list authdata/SDL_sdl.list data/Internal.mfs data/InternalPatcher.mfs data/ThinInternal.mfs

This almost completes our dataless server. Just remember that all the clients which connect to the server must have exactly the same content.

The last step for dirtsand to configure correctly is for you to provide the 'sdl' and 'ages' files to the client in unencrypted format;

Download and unzip 'Python27' from https://github.com/H-uru/moul-scripts/tree/python27 and extract the 'Python' and 'SDL' contents to the two folders as below.

/opt/dirtsand/ages/
/opt/dirtsand/SDL/

Don't encrypt them. Now you must change their ownership otherwise you'll be back at this step later! ; )

Open a terminal (it opens in your HOME directory by default) and type:

$ sudo chown -R dirtsand /opt/dirtsand

The -R flag will make sure "chown" changes the ownership of all files... recursively.

On Windows - SDL and Python

Now, move over to your Windows box.

The two tools you will use is plPythonPack.exe that was compiled earlier on and PlasmaCrypt.exe in your 'Plasma_Win32' (libhsplasma) download.

NOTE: Make sure to use the Win32 version of libhsPlasma to avoid any potential errors creeping in.

You will need to do a few things;

1. Copy the Dirtsand.keys and dirtsand.ini file over to your Windows box now, and place them in their own folder for easy reference.
2. Download libhsplasma from http://libhsplasma.googlecode.com/files/Plasma_Win32_r848.zip and unzip it to a safe place.
3. Setup a PATH variable to your unzipped Plasmatools (libhsplasma) folder and reboot your WinXP box.
4. unzip a fresh copy of moulscripts from https://github.com/H-uru/moul-scripts/tree/python27 and unzip it.
5. copy plPythonPack.exe and python27.dll into the unzipped 'Python27' folder.
6. Launch "cmd" and browse to the "\Python27\" folder
7. in a windows terminal type...

plPythonPack.exe Python

CAUTION: I tried to originally compile the client on Win7 64bit machine. There were some major errors I could not solve. As soon as I compiled a fresh build on a WinXP 32 bit system, everything fell into place.

You should notice one new file inside the "\Python27\" folder named python.pak.

\Python27\Python\python.pak

Although we have just packed the Python folder into "python.pak" it is still not encrypted.

To encrypt this file simply use "cmd" to navigate to the 'Python27\' folder.

Now, the files that need to be encrypted are the python.pak file you created earlier as well as all the "SDL" files.

Navigate to the '\Python27\Python\' folder and type (drop the inverted brackets in the command below):

prompt:> PlasmaCrypt droid -key {paste the key.droid value from ".\CWE Sandbox\INI Files\dirtsand.ini" here} python.pak

It should look like;

prompt:> PlasmaCrypt droid -key 86756....blah.blah...283746 python.pak

This 'key.droid' must be 32 digits long from your 'dirtsand.ini' file.

Next you need to encrypt the contents of the 'SDL' folder. Look at the download 'Sandbox Scripts' for examples. I dont guarantee they work for your setup, so use at your own risk.

NOTE: Unfortunately there is no recursive support in PlasmaCrypt, so either do each file in each folder one by one (and take a week to do it!) or use the spreadsheet technique below or use the "Sandbox Scripts". The technique below is a little time consuming (about an hour per spreadsheet), but you will only have to do it once.

We are going to create a spreadsheet with file listings for the contents of the 'SDL' folder that will be used to create to a batch file.

Once done you will save the spreadsheet as a normal file, then your final list of commands in each batch file should look like:

Example of sdl-enc.bat

PlasmaCrypt droid -key <key here> Ahnonay.sdl
etc
etc
etc

If you want to do this yourself you can build a list of files using;

dir -b > sdl-enc.csv

If you do create it yourself, this command will pipe its output to a file, so that you can edit it easily in scalc or excel easily.

Remember to:

1. Add YOUR droid key to spreadsheet
2. save as "csv" with space seperator
3. edit with scite or notepad, and remove all spaces and inverted commas
4. save the file with a "bat" extension into the SDL folder
5. check the format against the example above, sdl-enc.bat
6. use the cmd terminal and browse to the SDL folder and execute the batch file.
7. If you did it right, when you open any one of the SDL files in scite, it will look like gibberish with the starting line, "NotTheDroids"

Now the difference between a dataserving shard and a non-dataserving shard is as follows; Your newly encrypted python.pak and SDL files go in the root of your MOUL windows installation (not your server).

On Windows - Building a new MOUL install

Setting up your Windows Tiny Sandbox Client:

If you want to setup your Windows client from scratch with your compiled version of plClient and your new server.ini file, follow these steps:

Install MOULInstaller887.exe with your internet connection unplugged.

Watch out when the installer finishes... it tries to connect to Cyan, so make sure you got your network cable unplugged temporarily. You don't want their updates for your shard! The reason is, Cyans content is based on python 2.3, and your shard only runs python 2.7.

Then in the root of your fresh Sandbox installation do this:

• Delete UruLauncher.exe
• Delete UruExplorer.exe
• Copy your compiled plClient.exe to the root of your Tiny Sandbox Client
• Copy your SDL folder and Python folder to the root of your new installation. (There should only be one file in the Python folder, namely python.pak)
• Copy wrap_oal.dll and OpenAL32.dll from a defacto Cyan MOUL installation
• Copy NxCharacter.dll, NxCooking.dll, NxExtensions.dll and PhysX_Loader.dll from C:\Program Files (x86)\AGEIA Technologies\AGEIA PhysX SDK\v2.6.4\Bin\win32
• Copy python27.dll from C:\cwe-prefix\dll to the root of your client
• Copy resource.dat from http://www.guildofwriters.com/tools/resource.dat to the root of your client
• Edit tos.txt with the GNU GPL license
• If you dont have the correct version of PhysX download and install it from http://www.nvidia.com/object/physx-9.10.0513-driver.html
• Create a shortcut to plClient.exe and add -LocalData to the end of the target field, example;

<per> "C:\..\..\..\plClient" -LocalData </per>

Then to setup the server.ini file...

• Copy the Zrax's ini file at http://moul.zrax.net/zraxnet.ini and save it to your Tiny Sandbox.
• Rename zraxnet.ini to mynet.ini
• Open a Windows Command Prompt, change to that directory and run the following command:

Prompt:> PlasmaCrypt decrypt mynet.ini

Once the command is finished, open the file with a text editor (Use one that keeps format in place - Like Scite or Scintilla, just easier than notepad)

Open your Dirtsand.keys file and paste your 6 Client auth keys that were generated from dirtsand over the 6 Client auth keys in the mynet.ini file. If you open the mynet.ini file on your windows box and see mumbo jumbo, you did not decrypt your file correctly, so try the decrypt command again.

Now the last entries in the mynet.ini file are:

   Server.Gate.Host "xxx.xxx.xx.xxx" (IP address assigned to Ubuntu -

       - to find this go to your ubuntu terminal and type ifconfig, and use the address after "inet addr:")

   Server.Auth.Host "xxx.xxx.xx.xxx" (same as above)

   Server.DispName "The journey continues..." (whatever catch phrase kicks your hair back)

   Server.Status "" (ignore)

   Server.Signup "" (ignore)

Once done you will need to encrypt the mynet.ini file with the 'xtea' parameter in PlasmaCrypt. Just run the command:

Prompt:> PlasmaCrypt xtea mynet.ini

The command returns a success if all went well, copy mynet.ini to your Tiny Sandbox installation and rename it to 'server.ini'

On Ubuntu - Running Tiny Sandbox Server

Assuming everything else went smoothly, you should now be able to start your server and connect to it!

To start the server... You will use this command every time you want to start dirtsand after a reboot.

<per>$ bin/dirtsand dirtsand.ini </per>

You'll have to create an account for your FIRST PLAYER after this command is run, which can be done from the console:

$ ds-902> addacct <username> <password>

NOTE: The first time you run this can be quite odd since no vault is yet created. It seems to error out. When it happens just press enter and you should get to the dirtsand prompt.

If you need to verify that an account has been made, or you want to check that your server is listening for your windows client, you can follow the pgadmin and wireshark steps here...

Leave dirtsand running...

On Windows - Logging into the Sandbox Server

10) Run the Client: On Windows...

Launch the shortcut to plClient and enter the username and password you created in the step above.

After the license screen CWE should load and you will be taken to the menu to create an avatar. You know the rest.

Happy Exploring!

Additional Information

Download links:

• http://github.com/H-uru/dirtsand
• http://github.com/H-uru/Plasma
• https://github.com/H-uru/moul-scripts
• http://guildofwriters.com/tools/devlibs.zip