Monday, October 28, 2013

Ryzom Core Updates

It's been a long time since we've posted a meaningful update on the state of Ryzom Core.

Summer of Code Wrap-Up

We had another very successful year with Summer of Code. This year we mentored three great students: Quitta, fabien, and dfighter. All three students easily passed.The recap the projects:

Account Management System
Quitta took on the task of creating a new account management system for Ryzom Core. This tool fills a vacancy in the platform that is accomplished using a closed-source tool within the live Ryzom game. Quitta's tool allows players to register and edit their profile information as well as open and manage support tickets. It has a rich system for CSRs and admins to handle tickets and player accounts. It is also written to be module so that you can integrate Ryzom Core with any CMS authentication system - and uses a sync system to keep the CMS and Ryzom Core up to date with each others users. His code is complete and in theory production ready. It will be used shortly for the Ryzom Core development shard.

Programmable Shader Pipeline
dfighter took on yet another monumental task for his Summer of Code project. This year he took on the task of upgrading the Ryzom Core GL driver, implementing a programmable shader pipeline, converting the material system to work with the new driver and pipeline and finally creating an editor for Object Viewer Qt. He has some great videos on his progress, showing both the editor as well as the new driver in action. His code will hopefully be merged into default as some final lingering issues are dealt with.

Environmental Dependent Camera Animation
Fabien wrote a new camera effect system that allows content designers to tie camera effects or animations to other animations. It functions in some ways similar to the sound animation system. The example scenario that was developed was centered around the idea of a very large animals footsteps shaking the camera in relation to the foot-falls as well as the proximity to the listener - meaning the closer the large animation the greater the effect of the foot-fall camera shake. His code is basically complete but needs thorough testing before being merged to default.

World Editor Qt

Back in 2011 one of our community members and then-GSoC-student dnk-88 took on the task of rewriting the legacy World Editor tool into a Qt plugin for OVQt. His project was essentially complete and requires some minor functionality, testing, and bug resolution. Unfortunately we left his project sitting in its original branch at the end of Summer of Code that year and no one ever attempted to thoroughly test or otherwise use it.

We decided in order to try and drive some adoption of the plugin and start getting feedback on it we would merge it into default - it has some bugs and it needs a couple things completed on it but we won't know the full scope of what needs to be done until real users begin using it and telling us what it is lacking. Please compile this plugin (you will need both Landscape Editor and World Editor) and let us know how well it works.

Thursday, April 11, 2013

Google Summer of Code 2013!

The big news this week is that Ryzom Core was accepted to be in Google Summer of Code again under the umbrella of the awesome WorldForge project, along side the very cool CEGUI project. You can see all of our ideas on the WorldForge ideas page.

If you have questions or are looking for advice on building your proposal you can usually find us at the discussion link above or on IRC at in #ryzom.

Tuesday, February 12, 2013

Heads Up! Big GUI Merge Coming!

Just a warning to all Ryzom Core developers that the GSoC 2012 GUI Editor fork will be merged back into default within the next day or two. This branch contains much more than just a GUI Editor plugin to OVQt - it required substantial refactoring of the interface_v3 code in order to make it modular and reusable.

To view the source that will be merged, please look at dfighter's repository:

This is being done in preparation for the 0.9.0 release and to mitigate merge conflicts that dfighter has been battling over the past few months.

Thank you for your patience and let us know if there are any problems!

Tuesday, February 5, 2013

Ryzom Core: The Data Is Everywhere!

Ryzom Core has huge amounts of data to facilitate the world and its general running. However the nature of Ryzom Core data often times confuses new users of the platform. Many see the process of create the MySQL database and without fault assume that everything is persisted there. In reality very little data is actually retained in MySQL as the Ryzom Core platform was developed in a time before MySQL could be trusted with the workload live Ryzom required.

 So where is data stored in Ryzom Core? There are a plethora of types and locations. There are a lot of one-off scenarios as well but these are the major players:

  • MySQL for Shard Administration, WebIG apps and user accounting
  • Georges Sheets for static world data.
  • Primitives for world content and layout
  • PDR for persistence.

The MySQL Data Store

The MySQL databases are solely used by the services for general shard administration. This is where the shard services and domains are defined and how AS (Admin Service) knows what should be running and where. MySQL is also typically where any Ryzom API apps will stored their persistent data that isn't directly related to in-game data. For example a 'mailbox' app would store mail in a MySQL database more than likely.

When you sign up for a Ryzom Core account either through a custom signup form or through the user auto-generation that's enabled by default your user account and account privileges are stored in MySQL. This is just basic information like username and password, what shards you have access to and what privileges you have (such as GM or CSR.)

Georges Sheets

To understand what Georges Sheets are you have to understand what Ryzom Core means by "static data" versus world content. While primitives, discussed next, don't contain any persistent information they lay the world out in its base initial form. The Georges Sheets (which are compiled into packed sheets, a more complicated topic but essentially a compressed binary variation of the sheets) contain all of the static data that define things such as fauna, items, spells, and so on. These sheets define the characteristics that make up a mob or NPC but do not actually define a specific mob or NPC. This is the base data that you use to begin laying your world out.

Sheets also contain other static objects such as lists of titles, defining animations and animation sets, continents, starting player characteristics, and more. 

All of these are located in code/ryzom/common/data_leveldesign/leveldesign. You edit these sheets with the Georges Editor tool and they're essentially XML files with a parent/inheritance relationship capability.


While you can go into Georges Editor and create an NPC creature sheet that doesn't put that NPC anywhere on the map. In fact it doesn't necessary describe a specific NPC. Primitives, edited by the World Editor tool, define what the base version of the world, untouched by players, looks like. The layout defined in the primitives is what the world will look like when the shard is first started. 

To illustrate the differences between primitives and sheets when you add a new NPC you provide all of the details regarding that NPC including the ability to change its equipment and color. One of the things you do when defining an NPC is choose the sheet that defines them. This means that you create one NPC creature sheet as a template and then are able to layout actual NPCs with defined characteristics throughout the world.

In Ryzom Core you cannot place a specific mob (fauna) - as mobs and NPCs are distinctly different things. Instead you define what their patterns are and let the AI take over spawning, migration and so on. In the primitive you can configure this behavior but your primitive content will always reference a sheet which defines the basic characteristcs of the fauna as a template.

You can see the pattern now - Georges sheets provide the template, primitives provide the layout and structure and players provide the evolution. But since neither Georges or primitives store the world state that brings us to the next part.

All of these are located in code/ryzom/common/data_leveldesign/primitives. You edit these sheets with the World Editor tool and they're essentially an XML format (called Ligo).

PDR Data Stores

So if MySQL, Georges and Primitives aren't keeping track of characters, equipment, guilds and other such things that persist through shard reboots what is? The Ryzom Core supporting framework, NeL (Nevrax Library) has a rich and sophisticated serialization system. Nearly any object in Ryzom Core can be serialized into a stream to be transmitted over the wire or saved to a file. This means instead of having to translate a character object into MySQL insert/update statements Ryzom Core is able to simple pass the character object to a NeL file object and save it to disk or read it from disk.

The Ryzom Core service infrastructure takes this one step further with the PDR or Persistent Data Record system. This sophisticated system is able to replicate changes to other services in the shard and is maintained to disk through a master/slave backup service. This is the real source of runtime persistence in Ryzom Core. Where the data is stored is in the shards save_shard folder and depends on the type of data.

For example lets use characters. Characters are stored in code/ryzom/server/save_shard/characters in a pattern that the services determine. The actual file names are something like code/ryzom/server/save_shard/characters/001/account_1_0_pdr.bin where the 001 is simply a mechanism the services use to break up accounts into multiple folders (so you don't end up with 100k files in a single folder), the other two numbers (1 and 0) refer to account ID and character slot, respectively.

You can view the contents of the PDR bins using the pdr_util. While not encouraged as this is typically internal Ryzom Core data you can extract a PDR into XML, modify it and then re-assemble it using this tool. This would be helpful if you were, for example, writing a script to mass update characters for some reason.

Wednesday, January 30, 2013

Revising The Roadmap

So Edoardo Barolo pointed out that he can't see the road map if we move off of Redmine. He's right but I think that the roadmap we have is useless. It was created as a dream when we released this project into the world of open source and was never touched or thought of again. I think we should create a new roadmap, rethink what we're doing with Ryzom Core and decide what we, as a community, want our future to be.

The old Roadmap has two versions - 0.9.0 and 0.10.0. The goal of 0.9.0 was to port Ryzom Core. I'd say that this has been sufficiently accomplished. We have clients and servers on Windows, Mac and Linux. We have someone even building Ryzom Core on a Raspberry Pi  and another community member porting the client to Android (and it's looking pretty good!) So lets called 0.9.0 closed. Lets plan on doing a tag and release. But lets look at what 0.10.0 will be and the version after that. When do we as a community think we've achieved a 1.0 release?

The old roadmap calls for the creation of cloud-based tools. We've already begun a mass exodus of the tools off of MFC and towards Qt, largely thanks to the hard work of dnk-88's Summer of Code projects. Does this continue to be the goal for 0.10? What do you as world creators and developers see as essential tools that are missing? Is there functionality missing in the existing tools? What tools being ported to be multi-platform are an absolute necessity?

Does 0.10 or 0.11 also include something other than tools? Is there some game-play initiative we want to conquer and implement? Or server management tools? What do you dream of Ryzom Core having?

Please discuss here!

Monday, December 17, 2012

Particle System Demonstration

One of our community developers, dnk-88, was helping out Botanic from the Tempest in the Aether project with some particle system development. Particle systems are one of the more powerful features in Ryzom Core but are one of the features that we don't have a ton of documentation on how to use.

The particle system that dnk-88 created for Tempest was for their force-shield dome on one of their continents. The video documents the process from empty particle system to complete dome effect. It's a really cool process to watch and gives you a thorough demonstration of the new particle editor.

Note: You will want to full-screen the video to see the details and text better.

Tuesday, August 28, 2012

Tutorial: Chatty NPCs

In Ryzom Core we have one main NPC that randomly chats to you. You can take a look at how he chats you up about the four freedoms but you might want some more explanation. This also isn't the only way to make your NPCs talk your players ears off so lets see how Chiang works and then maybe expand his capabilities a little bit...

Random Chatter Using Timers

This is accomplished using a timer_t0_triggered NPC group event handler and a start_of_state NPC group event handler. The logic here is pretty simple: you use the start_of_state to set your t0 timer. In the case of Chiang the Strong it is set to 1000. This timer value uses in game ticks which is an important distinction. Many people would assume it is seconds or milliseconds. In timing terms there are 10 game ticks per real second so a value of 1000 is approximately every 100 seconds.

So now you have a timer that will go off in 10 seconds. You need to tie that timer to something. In the same NPC group you'll add another NPC group event handler to handle the timer_t0_triggered event. So now when your timer, created above, goes off it will get caught by this new event handler. In this event handler you need to do two things: set the timer again so he keeps chatting and then randomly say something. In order to accomplish two things your first action will be a multi_action which just allows you to specify more than one npc_event_action object. It will execute all of these actions.

The first action you'll add is a new set_timer_t0 action, it is set up the same way as your initial start_of_state but in the case of the Ryzom Core example we change the duration to 1500 - so he's a little less chatty. This new action sets the chatter loop in motion.

Next we'll add a random_select action. Here's a quick hint - most of these types are documented and you can see the local documentation in your favorite browser by clicking "View" next to the action drop down. You'll see this pattern throughout the classic World Editor tool. The documentation for random_select states the random choice is based on weight. This isn't true, it doesn't work. If you need it to work please log a bug. Under the random selection action you'll see that Chiang says four things - this is because he chats about the four freedoms all day long. So just keep adding say actions to your hearts content and he'll keep adding new things to babble about. You'll see that the say parameters are formatted like this:

say: CHIANG_1

You have a number of options in the say action parameters: tell, say, shout, universe, escort. I'm not really going to go into detail on these as for the most part they make sense. If you want your NPC to shout something you would use "shout" instead of "say". The second component, the CHIANG_1 is a phrase ID. Lets say you want Chiang to also say "Hey, by the way, Affero GPL is the best!" you would add a new say action with a phrase ID of something like CHIANG_5 or maybe CHIANG_AFFERO since the phrase ID is some what arbitrary. Let us assume you add CHIANG_5. Add your new action and set that for the parameter. Then you need to actually define that phrase somewhere. The details of translation are for another tutorial but to cut to the chase you would modify the phrase_XX.txt (where XX represents your various supported languages, e.g. en or fr) and add a new line there. It should be present in the phrase files on both the server and the client.

That's it, now Chiang says one more thing.

Mixing It Up

You can see how we can use a timer to quickly add repeatitive chatting but really you can use almost any NPC event. For example you could modify the NPCs In Motion tutorial so that the begin_state event handler has a multi_action where one action says a phrase like "Alright, lets get going" and the next action starts the guards in motion on the path.

Another popular alternative is to use the player_target_npc event handler - actions in this event handler get called whenever a player targets the NPC. When using this event handler type you have a player component and so instead of using the "say" parameter you could use the "tell" parameter.