News Forums Troubleshooting Serialization and Deserialization

This topic contains 12 replies, has 3 voices, and was last updated by  Sigil 3 months, 1 week ago.

Viewing 13 posts - 1 through 13 (of 13 total)
  • Author
    Posts
  • #33051

    jiri.navratil
    Participant

    Hi Rival Theory!

    First of all, thanks for your great library, which I trully find extremely powerfull and well-made.

    But I must admit there´s still something to dig around and that is serialization. If I look at your component XML import and export feature, you are throwing away serialized objects and custom data, which leads to reference loss on mount points(sensors, aspects and so on), AIRig body, EntityRig form etc. This can be solved by making custom xml and passing theese objects information and customData to DeserializeInPlace function. That leed mount points to properly load. But only in case of referencing game object in scene. Off-scene objects(prefabs) are not handled properly and all mount points references are loaded the wrong way. The think is, that it shift object references and that leads to assigning bad objects to mount points. In general, RAIN serialization is not capable of holding reference on prefab or asset. Also, AiRig memory is not loaded properly if you don´t clean up the memory first. I made to solve many cases, but I still haven´t managed to correctly serialize components, that have prefab references in them. This problem is most serious with behaviour trees. If you have for example 2 behaviour trees, the object is using one of them, you save the component and after some time, conditions has changed, object is using the 2nd one tree and if you load component to its last state, the first behaviour tree is not loaded(assigned). I am using dynamic composition of objects and I need to be able to construct them according to some scenarious, which specify, how is the object composed and what behaviour trees is using(without editor). If you dont select behaviour tree throw the editor, there is probably no way, how to achieve it.

    Finally, my question is, am I right or is there some workaround to achieve saving/loading prefab references correctly.

    PS: I am using UnitySerializer, which marks game objects with UniqueIdentifier and I can determine in runtime, whether the object is an instance of prefab or not. Second way could be attach script to object, that assign isPrefab variable to object and set it to true by default. Since off-scene objects dont have their awake function called, you can assign false to isPrefab in this function. That would lead to capability of determing, which gameobject came up from prefab and which not(in runtime).

    #33056

    prime
    Keymaster

    Just to be clear - the XML Import/Export function is not strictly “RAIN Serialization”. RAIN uses its own serialization code separate from Unity for everything, and it does correctly manage scene objects.

    Serializing scene objects in our XML exports is more complicated, and we don’t currently have a workable solution that is robust enough for us to release.

    What’s the main use case for what you are trying to accomplish?

    #33073

    jiri.navratil
    Participant

    I am trying to achieve serialization/deserialization of the scene using unity serializer and I ended up testing RAIN serialization. While looking at deserializeInPlace method, the 2nd argument is list of linkedObjects used in serialization. Theese are objects, that are references within the component(mount points, body, form etc.). If it is a prefab, nothing is added to the list and if you use deserialize, all references is shifted. For example. AIRig with body reference on gameobject in scene and 2 aspects(1st one with prefab reference, 2nd with other game object). If you use serialize/deserializeInPlace, the resulting component state is not valid. It results in correct body reference,but 1st aspect was prefab reference. That is lost and assigned with next reference from 2nd aspect. 3rd aspect would have the next ref, but there is no more linked object in list, so it results in “None” and reference is null. That is the biggest problem I stumbled across. I need to achieve scene serialization + deserialization(in editor and runtime). I am using Unity serializer. That correctly serialize scene objects, but it cant handle RAIN components. I wrote decorator for it to handle the serialization of RAIN components. But I still fight with it. I correctly save linked objects and customdata from serialization, so I am not loosing some of it on my own. Using null as 2nd and 3rd argument of deserializeInPlace method results in what I dont want - usage of existing data. In future, scene objects are composued dynamically. They are instantiated from prefabs with some default state. Then in runtime, they are enriched with AI components and some other controllers according the requirements on the usage and behaviour of the object based on some scenario definitions. For now on, I can not solve RAIN component deserialization to be able to continue my work.

    Also I wonder how is serialization/deserialization done within Unity editor. It correctly serialize/deserialize all objects including RAIN components. But it can not be used in runtime. There is no save method in runtime and when I use Unity serializer, the format is not correct and Load fails due to some corrupted data and reflection problems.

    • This reply was modified 3 months, 2 weeks ago by  jiri.navratil.
    #33075

    jiri.navratil
    Participant

    EDIT: I couldn´t edit the post once more. I overlooked typo at the beginning of sentence: “3rd aspect would have…” It is of course: “2nd aspect would have …”… The first case doesn´t make sense according my example.

    #33081

    prime
    Keymaster

    Looking into it.

    #33199

    Sigil
    Keymaster

    Sorry it took so long to get back to this.

    There was a little bit of confusion when I first saw this because I kept thinking Unity Serializer was referring to the Unity Editor’s serialization (my mistake). Once I saw what Unity Serializer really was I think I have a good idea of what you are trying to do.

    If I’m getting this right, you are trying to save your scene state after playing for awhile for a save file or similar. So RAIN’s serializing only occurs in the editor, and it’s deserializing only occurs when Awake gets called on the object (normally at least). To get the results you want, what I think you need to do is call Serialize() on any RAINComponent in your scene before saving it out. RAINComponent is our base MonoBehaviour that all of our objects inherit from (at least all the ones with custom serialization). This should save the current state of the AI into the component allowing it to be saved properly.

    Now when Unity Serializer restores the scene I don’t know when it loads the saved data, if it occurs after Awake is called on the RAINComponents you may need to manually call deserialize (RAINComponent.DataSerializer.DeserializeRAINObject(RAINComponent)).

    Let me know if you have any trouble and I’ll get back to you.

    #33203

    jiri.navratil
    Participant

    Hi Sigil,
    thank you for response. I am aware of when the serialization and deserialization occurs. I didn´t express myself clearly. I already call manually Serialize to reflect actual state of component to the DataSerializer. The problem lies somewhere else. What I am trying to achieve is mainly deserialization in runtime. With my actual solution, almost everything is fine except loading prefab references. As I already tried to explain, prefab references(for example not the ones in the current scene or behaviour trees) are not saved. From what I understand, RAIN custom serialization saved references on used objects to the collection called SerializedGameObjects. Every reference on scene objects is saved here properly except off scene objects. That leads to incorrect deserialization. For example I create entity rig with 3 aspects. The First one and the last one have mount points referenced on scene object. The 2nd one has mount point reference on prefab. When you call Serialize method, it reflects current state of component to its DataSerializer object. But, If you look at collection called SerializedObjects, there is only 2 objects. The ones, that is referenced within aspects and are in the scene. The prefab reference is lost. So when you use deserialization, all references listed after the prefab reference are shifted. On the aspects example, that leads to: 1st aspect has correct mount point reference. 2nd aspect has mount point of 3rd aspect, because there is missing one refence in SerializedObjects. And 3rd aspect dont have any reference in mount point. What I tried is to save information about objects in my custom xml. There is list of all components on testing RAIN object called puppet. I store serialized data, custom data and information about serialized objects from the SerializedObjects collection in DataSerializer. Because there are missing references, I am unable to get it working properly. By the way, The serialized data is encoded in base64, because there is used nested tags with same name and C# XmlDocument can not handle this(example - <tagname> <tagname/> </tagname>).

    Here is testing code I use. Make a gameobject called RAINPuppet1. Add any RAIN component, fill it with some data and mainly, use some prefab reference(not instantiated object from prefab!) as mount point. For example in aspects, sensors, behaviour etc. You can also clearly see it with behaviour tree reference - it is lost in Mind pane of AIRig component. You will see what I stumbled across.

    Thanks for your help!!!

    http://pastebin.com/DMGHAsYc

    • This reply was modified 3 months, 1 week ago by  jiri.navratil.
    #33213

    Sigil
    Keymaster

    Thank you for the code, that helped clarify.

    The first thing I see regarding prefabs is the way the object list is being saved out here:

    
    objectDataEntry.SetAttribute("type", o.GetType().ToString());
    objectDataEntry.SetAttribute("name", o.name);
    

    You also probably want to save its asset path (it should be blank if it isn’t an asset):

    
    objectDataEntry.SetAttribute("path", AssetDatabase.GetAssetPath(o));
    

    And then on load you would need to add a check before searching game objects:

    
    string name, type, path;
    name = type = path = "";
    XmlAttributeCollection noAttrs = no.Attributes;
    foreach (XmlAttribute a in noAttrs)
    {
        switch (a.Name)
        {
            case "name":
                name = a.Value;
                break;
            case "type":
                type = a.Value;
                break;
            case "path":
                path = a.Value;
                break;
            default:
                Debug.Log("unknown object entry attribute " + a.Name);
                break;
        }
    }
    if (path != "")
        objectsData.Add(AssetDatabase.LoadAssetAtPath(path, System.Type.GetType(type)));
    else
    {
        // Your other game object finding code
    }
    

    Untested code so be warned. This may not solve all the problems but it should help with the prefabs.

    #33216

    jiri.navratil
    Participant

    Thanks for the update.
    I know I would save the asset path and I would have done that if the asset object was in the collection SerializedGameObjects of DataSerializer object. But it is not saved with RAIN serialize function, that serializes the component and saves references on objects that are referenced within the component. The only thing I could do is go through all items of components recursively and check, if it is reference with reflection and based on that, save objects on my own. In addition, the deserialization of RAIN counts with right order of the SerializedGameObjects collection. That means, if I can´t insert the asset object at right place, it will shift all asignments and other properties(mount points etc.) would have incorrect references(on different object instead of the one it actually had assigned). Your solution would help only, if the DataSerializer.SerializedGameObjects contains the object or information about the asset, that was referenced. Isn´t there any workaround to achieve this?

    To clarify the deserialization without forcing you to test it on your own, I´ll make an ilustrated example.

    AIRig component - Body reference on GameObject with name “MyRootObject”
    - memory pane - one variable: name - “myobject”, value = “reference on GameObject with name “My1stObject”
    - mind pane - one reference on behaviour tree called “MyTree” ( this is asset!!!)
    - senses pane - 2 audio sensors - 1st one has mount point reference on asset “MySoldier”
    - 2nd one has mount point reference on “My3rdObject”

    Now, when the deserialization occurs, the SerializedGameObjects collection does not contain MyTree and MySoldier. And the following state of component occurs after deserialization:

    Body reference was on scene game object “MyRootObject” - correct
    in memory pane - one variable with reference on scene game object “My1stObject” - correct
    mind pane - had reference on the asset “MyTree” - Lost - now has no reference, because “MySoldier” is not type of BTNode
    - this case is rarity - normally all references are shifted is there is some missing, but Behaviour tree cannot be common asset

    senses pane - 1st audio sensor had mount point reference on asset “MySoldier”, which is lost and now has reference on “My3rdObject”
    - 2nd sensor has no reference (None), because there are on other object references in the collection.

    Other simple example would be :
    AIRig component: Body reference on prefab called “MySoldier” and in memory pane, one variable reference on game object “MyObject”.
    After deserialization, it results in body having “MyObject” reference and variable has no reference.

    Please note, that this last example has almost no use case, assigning body on asset is meaningless. I just wanted to demostrate, what happens. For example with aspects or sensors, it has some meaningful use case.

    I hope that helps clarify what I meant by the deserialization reference shifting due to missing asset references.
    Thank you again. I really appreciate your help.

    • This reply was modified 3 months, 1 week ago by  jiri.navratil.
    • This reply was modified 3 months, 1 week ago by  jiri.navratil.
    • This reply was modified 3 months, 1 week ago by  jiri.navratil. Reason: added some clarification
    #33240

    Sigil
    Keymaster

    Alright, I think I get what is going on, and hopefully I can help you track the problem down:

    1) When you are saving out the rain objects in your scene (by calling Serialize on the rain component) it is not saving any of the assets. Or at least when you look at SerializedGameObjects they aren’t in the list.

    2) Since you are missing these objects, when you call DeserializeInPlace your list of objects aren’t correct and so the rain object gets confused. (just to note here, if you happen to know the object is missing you can put null in to keep the fields lined up, but I think you are saying that you can’t tell it is missing, that it just isn’t there)

    So my confusion stems from #1, because in normal circumstances RAIN does save all of the assets into SerializedGameObjects. So if you are getting a list back that is shorter then it should be, and missing all of the assets, something else is happening.

    Just to be sure I wrote a little test to copy and paste an AIRig and I can not repeat what you are seeing with my objects. Let me know if I am still missing the real problem here. I’ll try to catch your reply before I go to sleep tonight so we don’t have to wait another day.

    
    using RAIN.BehaviorTrees;
    using RAIN.Core;
    using RAIN.Serialization;
    using System.Collections.Generic;
    using UnityEngine;
    public class TestSerialization : MonoBehaviour
    {
        [SerializeField]
        private bool _checkObjects = false;
        [SerializeField]
        private bool _copyRigWithAssets = false;
        [SerializeField]
        private bool _copyRigWithoutTrees = false;
    	void Update()
        {
            if (_checkObjects)
            {
                _checkObjects = false;
                AIRig tRig = GetComponent<AIRig>();
                tRig.Serialize();
                string tPrintObjects = "";
                for (int i = 0; i < tRig.DataSerializer.SerializedGameObjects.Count; i++)
                {
                    Object tObj = tRig.DataSerializer.SerializedGameObjects[i];
                    if (tObj == null)
                        tPrintObjects += "Object " + i + " is null\n";
                    else
                        tPrintObjects += "Object " + i + " is " + tObj.GetType().ToString() + "\n";
                }
                Debug.Log(tPrintObjects);
            }
            if (_copyRigWithAssets || _copyRigWithoutTrees)
            {
                AIRig tRig = GetComponent<AIRig>();
                tRig.Serialize();
                string tXML = tRig.DataSerializer.SerializedData;
                List<Object> tObjects = new List<Object>(tRig.DataSerializer.SerializedGameObjects);
                List<FieldSerializer.CustomSerializedData> tCustomObjects = new List<FieldSerializer.CustomSerializedData>();
                for (int i = 0; i < tRig.DataSerializer.SerializedCustomData.Count; i++)
                {
                    byte[] tOldData = tRig.DataSerializer.SerializedCustomData[i].Data;
                    byte[] tCopyData = new byte[tOldData.Length];
                    System.Array.Copy(tOldData, tCopyData, tCopyData.Length);
                    tCustomObjects.Add(new FieldSerializer.CustomSerializedData(tCopyData));
                }
                if (_copyRigWithoutTrees)
                {
                    for (int i = 0; i < tObjects.Count; i++)
                    {
                        if (tObjects[i] is BTAsset)
                            tObjects[i] = null;
                    }
                }
                AIRig tNewRig = gameObject.AddComponent<AIRig>();
                tNewRig.DeserializeInPlace(tXML, tObjects, tCustomObjects);
                _copyRigWithAssets = false;
                _copyRigWithoutTrees = false;
            }
    	}
    }
    
    • This reply was modified 3 months, 1 week ago by  Sigil. Reason: Fixed the custom data copy so that it actually copied
    #33246

    jiri.navratil
    Participant

    Hi Sigil,

    You are correct, with this approach, it is deserializing correctly. The problem have to lie in saving information about objects into file. What you did is saving serialized data into xml, which can be saved out into file, customData as well since it is collection of byte arrays, but how to properly save SerializedObjects collection into file? Do you have any ideas?

    I need to save whole scene into file and be able to load it based on some scenarios. This approach only allows me to store changes made in runtime and dont lose it when you switch between editor and play mode. It is great, but still not what I am trying to achieve.

    EDIT:
    Saving the collection is right. The problem lies in finding and refering to objects. I was using GameObject.Find to match existing objects in scene, which is not correct. That´s why I was losing references. Do you have any suggestions, how to correctly and clearly find objects and assets to handle this?

    Thank you very much!

    • This reply was modified 3 months, 1 week ago by  jiri.navratil.
    #33248

    jiri.navratil
    Participant

    Hi guys,
    I just managed to deserialize components properly including assets. The only problem which may lead to problem is when you need to find objects of given name and type or id. Some of mount points needs Transform component of object, some of them GameObject itself, some others… Most critical case would be custom sensor, aspect or element with reference on collider, rigidbody or at worst, custom type. Now where I see problem is, that if I iterate over all objects returned by Object.GetAllObjectsOfType<Object>(), match the one I need and add it to the collection SerializedObject(your tObjects), it has to be the proper object of type, that accept the property i am assigning the reference to. For example if “Mount Point” accepts reference on Transform, and I found the object matching criteria(unique id) that was referenced here, I have to pass to this property correct object… So not UnityEngine.Object, but (UnityEngine.Object o as GameObject).transform… And here lies the problem. How can I properly cast an object to unknown type. It can be almost anything… Vector3D, Quaternion, Transform, Joint, Collider, … there are so many built-in types and so many cases,when it is highly recommended and usefull to make custom types or wrappers on the buil-in ones.

    http://pastebin.com/cwg81PgD - line 313

    Does anything come up into your mind?

    Btw thanks a lot for your help. It helped me to clarify, what I did wrong and where another potencial problems lie =)

    • This reply was modified 3 months, 1 week ago by  jiri.navratil.
    #33254

    Sigil
    Keymaster

    I’m glad things are coming together. You are right, it is difficult to identify what the type is and how to get it, this is exactly why RAIN has that list to begin with, instead of tracking down what the object really is from a find or such, it is stored and pointed to by index.

    So before we go too much further, what is the intent of this? Early on you mentioned UnitySerializer (and perhaps some problems with integrating that and RAIN), is this still for that or are you attempting a different solution?

    The main reason I ask is this: How you are handling the storing of instantiated objects in your scene? Or any serialized object in the scene (other than RAIN)? Whatever is being used for that, what hurdles did you run into using it with RAIN?

    Or are you using a custom solution?

Viewing 13 posts - 1 through 13 (of 13 total)

You must be logged in to reply to this topic.