Friday, March 31, 2017

Development tutorial link: Extensibility challenges: Pack/Unpack in RunBase classes

Introduction + example

As you know, we have been focusing on extending our Extensibility story in the application, as well as trying to document the various patterns common to the application and how to address them if you are an ISV and need to extend some existing functionality.

mfp has recently written a blog post describing how you can extend the information shown on a RunBase-based dialog, and how to handle that information once the user enters the necessary data.

You can read through that particular example here: 
What that example did not describe is how to preserve the user entered data, so that next time the dialog is opened, it contains the last entries already populated. This is the typical pattern used across all AX forms and is internally based on the SysLastValue table.

In RunBase classes it is done through the pack and unpack methods (as well as initParmDefault).
For ensuring seemless code upgrade of these classes they also rely on a "version" of the stroed SysLastValue data, which is typically stored in a macro definition. The RunBase internal class state that needs to be preserved between runs is typically done through a local macro.
A typical example is shown below (taken from the Tutorial_RunBaseBatch class):


    public container pack()
        return [#CurrentVersion, #CurrentList];

    public boolean unpack(container packedClass)
        Version version = RunBase::getVersion(packedClass);
        switch (version)
            case #CurrentVersion:
                [version,#CurrentList] = packedClass;
                return false;

        return true;

Just in short, what happens is that:

  • We save the packed state of the class with the corresponding version into the SysLastValue table record for this class, which means that all variables in the CurrentList macro need to be "serializable". 
    • The container will look something like this: [1, 31/3/2017, "US-0001"]
  • When we need to retrieve/unpack these values, we retrieve the version as we know it's the first position in the container.
    • If the version is still the same as the current version, read the packed container into the variables specified in the local macro
    • If the version is different from the current version, return false, which will subsequently run initParmDefault() method to load the default values for the class state variables 

Problem statement

This works fine in overlayering scenarios, because you just add any additional state to the CurrentList macro and they will be packed/unpacked when necessary automatically.

But what do you do when overlayering is not an option? You use augmentation / extensions.

However, it is not possible to extend macros, either global or locally defined. Macros are replaced with the corresponding text at compile time which would mean that all the existing code using the macros would need to be recompiled if you extended it, which is not an option.

OK, you might say, I can just add a post-method handler for the pack/unpack methods, and add my additional state there to the end of the container.

Well, that might work if your solution is the only one, but let's look at what could happen where there are 2 solutions side by side deployed:
  1. Pack is run and returns a container looking like this (Using the example from above): [1, 31/3/2017, "US-0001"]
  2. Post-method handler is called on ISV extension 1, and returns the above container + the specific state for ISV 1 solution (let's assume it's just an extra string variable): [1, 31/3/2017, "US-0001", "ISV1"]
  3. Post-method handler is called on ISV extension 2, and returns the above container + the specific state for ISV 2 solution: [1, 31/3/2017, "US-0001", "ISV1", "ISV2"]
Now, when the class is run the next time around, unpack needs to be called, together with the unpack method extensions from ISV1 and ISV2 solutions.

  1. Unpack is run and assigns the variables from the packed state (assuming it's the right version) to the base class variables.
  2. ISV2 unpack post-method handler is called and needs to retrieve only the part of the container which is relevant to ISV2 solution
  3. ISV1 unpack post-method handler is called and needs to do the same 

Steps 2 and 3 cannot be done in a reliable way. OK, say we copy over the macro definitions from the base class, assuming also the members are public and can be accessed from our augmentation class or we duplicate all those variables in unpack and hope nothing changes in the future :) - and in unpack we read the sub-part of the container from the base class into that, but how can we ensure the next part is for our extension? ISV1 and ISV2 post-method handlers are not necessarily going to be called in the same order for unpack as they were for pack.

All in all, this just does not work.


The below line is perfectly fine in X++ and will not cause issues, which is why the base unpack() would not fail even if the packed container had state for some of the extensions as well.

[cn, value, value2] = ["SomeClass", 4, "SomeValue", "AnotherClass", true, "more values"];

The container being assigned has more values than the left side.


In order to solve this problem and make the behavior deterministic, we came up with a way to uniquely identify each specific extension packed state by name and allow ISVs to call set/get this state by name.

With Platform Update 5 we have now released this logic at the RunBase level. If you take a look at that class, you will notice a few new methods:
  • packExtension - adds to the end of the packed state (from base or with other ISV extensions) container the part for this extension, prefixing it with the name of the extension
  • unpackExtension - look through the packed state container and find the sub-part for this particular extension based on extension name.
  • isCandidateExtension - evaluates if the passed in container is possible an extension packed state. For that it needs to consist of the name of the extension + packed state in a container.
You can read more about it and look at an example follow-up from mfp's post below:

Hope this helps!