Google Summer of Code 2013!

The big news this week is that Ryzom Core was accepted to be in Google Summer of Code again under the umbrella of the awesome WorldForge project, along side the very cool CEGUI project. You can see all of our ideas on the WorldForge ideas page.

If you have questions or are looking for advice on building your proposal you can usually find us at the discussion link above or on IRC at irc.freenode.net in #ryzom.

Heads Up! Big GUI Merge Coming!

Just a warning to all Ryzom Core developers that the GSoC 2012 GUI Editor fork will be merged back into default within the next day or two. This branch contains much more than just a GUI Editor plugin to OVQt - it required substantial refactoring of the interface_v3 code in order to make it modular and reusable.

To view the source that will be merged, please look at dfighter's repository:

https://bitbucket.org/dfighter1985/ryzom-fork

This is being done in preparation for the 0.9.0 release and to mitigate merge conflicts that dfighter has been battling over the past few months.

Thank you for your patience and let us know if there are any problems!

Ryzom Core: The Data Is Everywhere!

Ryzom Core has huge amounts of data to facilitate the world and its general running. However the nature of Ryzom Core data often times confuses new users of the platform. Many see the process of create the MySQL database and without fault assume that everything is persisted there. In reality very little data is actually retained in MySQL as the Ryzom Core platform was developed in a time before MySQL could be trusted with the workload live Ryzom required.

 So where is data stored in Ryzom Core? There are a plethora of types and locations. There are a lot of one-off scenarios as well but these are the major players:

• MySQL for Shard Administration, WebIG apps and user accounting
• Georges Sheets for static world data.
• Primitives for world content and layout
• PDR for persistence.

The MySQL Data Store

The MySQL databases are solely used by the services for general shard administration. This is where the shard services and domains are defined and how AS (Admin Service) knows what should be running and where. MySQL is also typically where any Ryzom API apps will stored their persistent data that isn't directly related to in-game data. For example a 'mailbox' app would store mail in a MySQL database more than likely.

When you sign up for a Ryzom Core account either through a custom signup form or through the user auto-generation that's enabled by default your user account and account privileges are stored in MySQL. This is just basic information like username and password, what shards you have access to and what privileges you have (such as GM or CSR.)

Georges Sheets

To understand what Georges Sheets are you have to understand what Ryzom Core means by "static data" versus world content. While primitives, discussed next, don't contain any persistent information they lay the world out in its base initial form. The Georges Sheets (which are compiled into packed sheets, a more complicated topic but essentially a compressed binary variation of the sheets) contain all of the static data that define things such as fauna, items, spells, and so on. These sheets define the characteristics that make up a mob or NPC but do not actually define a specific mob or NPC. This is the base data that you use to begin laying your world out.

Sheets also contain other static objects such as lists of titles, defining animations and animation sets, continents, starting player characteristics, and more. 

All of these are located in code/ryzom/common/data_leveldesign/leveldesign. You edit these sheets with the Georges Editor tool and they're essentially XML files with a parent/inheritance relationship capability.

Primitives

While you can go into Georges Editor and create an NPC creature sheet that doesn't put that NPC anywhere on the map. In fact it doesn't necessary describe a specific NPC. Primitives, edited by the World Editor tool, define what the base version of the world, untouched by players, looks like. The layout defined in the primitives is what the world will look like when the shard is first started. 

To illustrate the differences between primitives and sheets when you add a new NPC you provide all of the details regarding that NPC including the ability to change its equipment and color. One of the things you do when defining an NPC is choose the sheet that defines them. This means that you create one NPC creature sheet as a template and then are able to layout actual NPCs with defined characteristics throughout the world.

In Ryzom Core you cannot place a specific mob (fauna) - as mobs and NPCs are distinctly different things. Instead you define what their patterns are and let the AI take over spawning, migration and so on. In the primitive you can configure this behavior but your primitive content will always reference a sheet which defines the basic characteristcs of the fauna as a template.

You can see the pattern now - Georges sheets provide the template, primitives provide the layout and structure and players provide the evolution. But since neither Georges or primitives store the world state that brings us to the next part.

All of these are located in code/ryzom/common/data_leveldesign/primitives. You edit these sheets with the World Editor tool and they're essentially an XML format (called Ligo).

PDR Data Stores

So if MySQL, Georges and Primitives aren't keeping track of characters, equipment, guilds and other such things that persist through shard reboots what is? The Ryzom Core supporting framework, NeL (Nevrax Library) has a rich and sophisticated serialization system. Nearly any object in Ryzom Core can be serialized into a stream to be transmitted over the wire or saved to a file. This means instead of having to translate a character object into MySQL insert/update statements Ryzom Core is able to simple pass the character object to a NeL file object and save it to disk or read it from disk.

The Ryzom Core service infrastructure takes this one step further with the PDR or Persistent Data Record system. This sophisticated system is able to replicate changes to other services in the shard and is maintained to disk through a master/slave backup service. This is the real source of runtime persistence in Ryzom Core. Where the data is stored is in the shards save_shard folder and depends on the type of data.

For example lets use characters. Characters are stored in code/ryzom/server/save_shard/characters in a pattern that the services determine. The actual file names are something like code/ryzom/server/save_shard/characters/001/account_1_0_pdr.bin where the 001 is simply a mechanism the services use to break up accounts into multiple folders (so you don't end up with 100k files in a single folder), the other two numbers (1 and 0) refer to account ID and character slot, respectively.

You can view the contents of the PDR bins using the pdr_util. While not encouraged as this is typically internal Ryzom Core data you can extract a PDR into XML, modify it and then re-assemble it using this tool. This would be helpful if you were, for example, writing a script to mass update characters for some reason.

Revising The Roadmap

So Edoardo Barolo pointed out that he can't see the road map if we move off of Redmine. He's right but I think that the roadmap we have is useless. It was created as a dream when we released this project into the world of open source and was never touched or thought of again. I think we should create a new roadmap, rethink what we're doing with Ryzom Core and decide what we, as a community, want our future to be.

The old Roadmap has two versions - 0.9.0 and 0.10.0. The goal of 0.9.0 was to port Ryzom Core. I'd say that this has been sufficiently accomplished. We have clients and servers on Windows, Mac and Linux. We have someone even building Ryzom Core on a Raspberry Pi  and another community member porting the client to Android (and it's looking pretty good!) So lets called 0.9.0 closed. Lets plan on doing a tag and release. But lets look at what 0.10.0 will be and the version after that. When do we as a community think we've achieved a 1.0 release?

The old roadmap calls for the creation of cloud-based tools. We've already begun a mass exodus of the tools off of MFC and towards Qt, largely thanks to the hard work of dnk-88's Summer of Code projects. Does this continue to be the goal for 0.10? What do you as world creators and developers see as essential tools that are missing? Is there functionality missing in the existing tools? What tools being ported to be multi-platform are an absolute necessity?

Does 0.10 or 0.11 also include something other than tools? Is there some game-play initiative we want to conquer and implement? Or server management tools? What do you dream of Ryzom Core having?

Please discuss here!

Particle System Demonstration

One of our community developers, dnk-88, was helping out Botanic from the Tempest in the Aether project with some particle system development. Particle systems are one of the more powerful features in Ryzom Core but are one of the features that we don't have a ton of documentation on how to use.

The particle system that dnk-88 created for Tempest was for their force-shield dome on one of their continents. The video documents the process from empty particle system to complete dome effect. It's a really cool process to watch and gives you a thorough demonstration of the new particle editor.

Note: You will want to full-screen the video to see the details and text better.

Tutorial: Chatty NPCs

In Ryzom Core we have one main NPC that randomly chats to you. You can take a look at how he chats you up about the four freedoms but you might want some more explanation. This also isn't the only way to make your NPCs talk your players ears off so lets see how Chiang works and then maybe expand his capabilities a little bit...

Random Chatter Using Timers

This is accomplished using a timer_t0_triggered NPC group event handler and a start_of_state NPC group event handler. The logic here is pretty simple: you use the start_of_state to set your t0 timer. In the case of Chiang the Strong it is set to 1000. This timer value uses in game ticks which is an important distinction. Many people would assume it is seconds or milliseconds. In timing terms there are 10 game ticks per real second so a value of 1000 is approximately every 100 seconds.

So now you have a timer that will go off in 10 seconds. You need to tie that timer to something. In the same NPC group you'll add another NPC group event handler to handle the timer_t0_triggered event. So now when your timer, created above, goes off it will get caught by this new event handler. In this event handler you need to do two things: set the timer again so he keeps chatting and then randomly say something. In order to accomplish two things your first action will be a multi_action which just allows you to specify more than one npc_event_action object. It will execute all of these actions.

The first action you'll add is a new set_timer_t0 action, it is set up the same way as your initial start_of_state but in the case of the Ryzom Core example we change the duration to 1500 - so he's a little less chatty. This new action sets the chatter loop in motion.

Next we'll add a random_select action. Here's a quick hint - most of these types are documented and you can see the local documentation in your favorite browser by clicking "View" next to the action drop down. You'll see this pattern throughout the classic World Editor tool. The documentation for random_select states the random choice is based on weight. This isn't true, it doesn't work. If you need it to work please log a bug. Under the random selection action you'll see that Chiang says four things - this is because he chats about the four freedoms all day long. So just keep adding say actions to your hearts content and he'll keep adding new things to babble about. You'll see that the say parameters are formatted like this:

say: CHIANG_1

You have a number of options in the say action parameters: tell, say, shout, universe, escort. I'm not really going to go into detail on these as for the most part they make sense. If you want your NPC to shout something you would use "shout" instead of "say". The second component, the CHIANG_1 is a phrase ID. Lets say you want Chiang to also say "Hey, by the way, Affero GPL is the best!" you would add a new say action with a phrase ID of something like CHIANG_5 or maybe CHIANG_AFFERO since the phrase ID is some what arbitrary. Let us assume you add CHIANG_5. Add your new action and set that for the parameter. Then you need to actually define that phrase somewhere. The details of translation are for another tutorial but to cut to the chase you would modify the phrase_XX.txt (where XX represents your various supported languages, e.g. en or fr) and add a new line there. It should be present in the phrase files on both the server and the client.

That's it, now Chiang says one more thing.

Mixing It Up

You can see how we can use a timer to quickly add repeatitive chatting but really you can use almost any NPC event. For example you could modify the NPCs In Motion tutorial so that the begin_state event handler has a multi_action where one action says a phrase like "Alright, lets get going" and the next action starts the guards in motion on the path.

Another popular alternative is to use the player_target_npc event handler - actions in this event handler get called whenever a player targets the NPC. When using this event handler type you have a player component and so instead of using the "say" parameter you could use the "tell" parameter.

Tutorial: Settings NPCs In Motion

In this post I'm going to describe to a simple way to set your NPCs into motion. We'll be using NPC routes.

To begin with open up World Editor and make sure that the urban_newbieland primitive is loaded. It would be exceptionally helpful to have the newbieland.land file loaded as well so that you can visualize the continent as you draw your paths and zones.

Creating an NPC Path

1. Navigate to rangers_starting_city (the npc_manager)
2. Right click on rangers_starting_city and choose npc_route
1. Provide a name, e.g. patrol_npcs
2. Set AI Activity to either guard or normal.
3. Set AI Movement to follow_route
4. Set vertical_pos to auto.
3. Ensure that patrol_npcs is selected
1. Enter Edit Point mode
2. Enable Add Points
3. Draw your path.
4. Right click on patrol_npcs and add a new NPC Group
5. Name it "guard_npcs"
6. Right click on guard_npcs and add a new NPC Bot
1. Name it newbieland_guard_patrol - ensure the name is unique if you add multiple bots.
2. Set the sheet_client, e.g. company_of_the_drill_loge_master_z_h_b1
3. Set vertical_pos to auto
4. You may also set chat_parameters if necessary and equipment if you want to customize equipment or colors. For reference see chiang_the_strong.
5. Move the point representing the bot(s) to somewhere near the beginning of the path.
7. Next you will have to enable the NPC event handling.
1. Right click on patrol_npcs and add a new npc_state_event_handler
2. Name it "destination_reached_all"
3. Set the event to destination_reached_all
4. Right click on the new event handler and add a new npc_event_handler_action
1. Name it "begin_state"
2. Set the action to begin_state
3. Set the parameters to patrol_npcs
8. Save the primitive and reboot your EGS and AIS. A full shard reboot is recommended to simplify potential problems during content development.