Monday, May 20, 2019

[Development tutorial] Sample test + tips for using ATL (Acceptance Test Library) to implement tests in Warehouse management

Hopefully, by now everyone is familiar with the fact that we shipped our internal test framework for all ISVs, partners and customers to use.
We have published pretty detailed documentation about the framework on docs.microsoft.com:

https://docs.microsoft.com/en-us/dynamics365/unified-operations/dev-itpro/perf-test/acceptance-test-library

I, however, learn much better when looking at samples. We have shipped a couple sample tests in each area with the 10.0.3 release.
In this post, I would like to share another sample test case and walk through the different types of commands possible in ATL, focusing also on the part that drives mobile device flow operations.

Important note

ATL is not meant as a replacement for good quality production code, and should not be used to execute business logic in PROD environments

Sample test

This test will validate a very simple inbound flow where I want to validate I can confirm the putaway location by the location check digit:
  • Create a new purchase order for a WHS-enabled item to a WHS-enabled warehouse
  • Perform the registration of the item through the mobile device
  • Perform the putaway through the mobile device
    • Confirm the location confirmation is shown and is asking for the check digit
    • Enter the check digit of the putaway location instead of the location ID
    • Complete the putaway
  • Validate the work lines are updated to Closed (not really the point of the check digit test, but just to show the capability)

[SysTestMethod]
public void confirmLocationWithCheckDigit_SimplePurchOrderOneItem_Success()
{
 ttsbegin;

 //========Given========

 purchaseOrder.addLine().setItem(item).setInventDims([warehouse]).setQuantity(PurchaseQuantity).save();

 rf.login().purchaseOrderItemReceiving().setPurchaseOrder(purchaseOrder).setItem(item).setQuantity(PurchaseQuantity).confirmUnitSelection();

 var work = purchaseOrder.work().singleEntity();

 //========When========

 var poPutaway = rf.login().purchaseOrderPutaway().setWork(work).confirmPick();

 poPutaway.parmCheckDigitControl().assertIsEnabled(); // Will automatically happen within set method below
 poPutaway.setCheckDigit(checkDigit); // This will confirm the Put to this location

 //========Then========

 poPutaway.assertWorkCompleted();

 work.reread();
 this.assertEquals(WHSWorkStatus::Closed, work.parmWorkStatus(), 'Work should be closed');

 work.lines().assertExpectedLines(
  whs.workLines().spec().withLineNum(1).withItem(item).withStatusClosed().withQuantity(PurchaseQuantity).withLocation(whs.locations().receipt()),
  whs.workLines().spec().withLineNum(2).withItem(item).withStatusClosed().withQuantity(PurchaseQuantity).withLocation(putawayLocation));

 ttsabort;
}

Looks pretty simple, doesn't it? And does not really require comments, as the code is pretty self explanatory for the most part, "even a functional consultant can read it" :)
A number of things I would like to call out here:
  • Notice the name of the test method. We always try to name the method according to the Given/When/Then pattern (also explicitly called out in the method body), more specifically, When_Given_Then(). Or, for those more familiar with AAA, Arrange_Act_Assert()
    • Note the method name does not start with test. You can still do that, but we prefer being explicit through decorating the method with the SysTestMethod attribute
  • I skipped the test setup part - I'll post that below. Normally that would include steps generic for all tests, while the Given part in here would only contains a few things special to this particular test. In this test, we 
    • create a purchase order line for a small quantity of a WHS-enabled item
    • do the PO receipt through the mobile device, to create the putaway work.
  • There is A LOT of defaulting that happens in these fluent calls. When writing code, you need to pay attention and override any default value. For example, 
    • When setting inventory dimensions for the PO line, I only specify the warehouse, but that in turn sets the Site as part of the business logic. The Inventory status is also defaulted based on the Warehouse parameters. 
    • When logging it to the mobile device in rf.login(), the default worker is used. However you can overwrite that and specify another worker to log in with. 
  • The WMA flows should look like you'd have it on the actual device, with scanner actually sending a caret return aka clicking OK after entry. Note, that defaulting is also supported here, where we just proceed without explicitly specifying a unit
    • Here, however, I used a dedicated method, confirmUnitSelection(), to improve readability. Inside, in the implementation, it's a simple OK button click. 
    • It is possible to override this behavior, by disabling the automatic submission on the flow class. This way, you'll have the ability to for example set multiple controls at the same time before OK'ing, etc. 
  • ATL allows for most areas to rely on either a special entity class, or the underlying record. Generally speaking, we always prefer to use the entity over the record. For example, take a look at how I retrieve the work entity for the putaway work created after registration.
  • Process Guide flows are tested with a different set of classes at this point. We'd be interested to hear your feedback on, which approach is more convenient. 
  • When doing putaway, I also use a "dummy" method, confirmPick() to confirm the pick up of the LP from the RECV location. These methods, however, could be doing some additional validation
    • We have, for example, implemented some like confirmPutToBaydoor(), which will confirm we are in fact in a Put step, and the Location control has a value where it is of type "Final shipping location". Think through these scenarios when extending ATL
  • We can also access the controls shown on the current screen right now, to perform additional validation, for example. I showcase that with the CheckDigit control, simply checking that it is enabled for data entry.
    • This already happens automatically in the next call to setCheckDigit(), so you should consider where you really need this.
    • assertIsEnabled() will also ensure the control is in fact visible.
  • Lastly, I wanted to show how specs are used in a real example, where we in assertExpectedLines() specify the 2 lines we expect to see on the putaway work after execution of the putaway. This allows for really flexible validation of the data, that also is easy to read and understand.
  • Check out how the flow class shows up in the debugger - it displays the XML with the controls, which really helps understand where you are in the flow if something in the test goes very wrong!

Entire test class

Below is the full listing of the xpp class with the test, including variable declarations and setup.
You can also download it from my OneDrive

[SysTestGranularity(SysTestGranularity::Integration),
SysTestCaseAutomaticNumberSequences]
public final class DEV_PurchaseOrderReceivingAndPutawayScenarioTests extends SysTestCase
{
    AtlDataRootNode                     data;
    AtlDataInvent                       invent;
    AtlDataInventOnHand                 onHand;
    AtlDataWHS                          whs;
    AtlDataWHSMobilePortal              rf;
    AtlDataPurch                        purch;

    InventTable                         item;
    InventLocation                      warehouse;
    WMSLocation                         putawayLocation;
    AtlEntityPurchaseOrder              purchaseOrder;

    private const WMSCheckText          CheckDigit = '12345';
    private const PurchQty              PurchaseQuantity = 1;

    public void setUpTestCase()
    {
        super();

        data = AtlDataRootNode::construct();
        invent = data.invent();
        onHand = data.invent().onHand();
        whs = data.whs();
        rf = whs.rf();
        purch = data.purch();

        ttsbegin;

        whs.helpers().setupBasicPrerequisites();
        AtlWHSWorkUserContext::currentWorkUser(whs.workUsers().default());

        item = invent.items().whs();
        warehouse = whs.warehouses().whs();
        whs.warehouses().ensureDefaultReceiptLocation(warehouse);

        putawayLocation = whs.locations().floor();
        putawayLocation.checkText = CheckDigit;
        putawayLocation.update();

        whs.locationDirectives().purchasePutaway();
        whs.workTemplates().purchaseReceipt();

        var poPutawayMenuItem = rf.menuItems().purchaseOrderPutaway();
        rf.menuItems().setupWorkConfirmation(poPutAwayMenuItem.MenuItemName, WHSWorkType::Put, NoYes::No, NoYes::Yes);

        rf.menus().default().addMenuItem(rf.menuItems().purchaseOrderItemReceiving())
                            .addMenuItem(poPutawayMenuItem);

        purchaseOrder = data.purch().purchaseOrders().createDefault();

        ttscommit;
    }

 [SysTestMethod]
 public void confirmLocationWithCheckDigit_SimplePurchOrderOneItem_Success()
 {
  ttsbegin;

  //========Given========

  purchaseOrder.addLine().setItem(item).setInventDims([warehouse]).setQuantity(PurchaseQuantity).save();

  rf.login().purchaseOrderItemReceiving().setPurchaseOrder(purchaseOrder).setItem(item).setQuantity(PurchaseQuantity).confirmUnitSelection();

  var work = purchaseOrder.work().singleEntity();

  //========When========

  var poPutaway = rf.login().purchaseOrderPutaway().setWork(work).confirmPick();

  poPutaway.parmCheckDigitControl().assertIsEnabled(); // Will automatically happen within set method below
  poPutaway.setCheckDigit(checkDigit); // This will confirm the Put to this location

  //========Then========

  poPutaway.assertWorkCompleted();

  work.reread();
  this.assertEquals(WHSWorkStatus::Closed, work.parmWorkStatus(), 'Work should be closed');

  work.lines().assertExpectedLines(
   whs.workLines().spec().withLineNum(1).withItem(item).withStatusClosed().withQuantity(PurchaseQuantity).withLocation(whs.locations().receipt()),
   whs.workLines().spec().withLineNum(2).withItem(item).withStatusClosed().withQuantity(PurchaseQuantity).withLocation(putawayLocation));

  ttsabort;
 }

}

A few extra notes:

  • We recommend that you create a base test class for your test automation, that will hide the ATL navigation variable declaration and initialization, along with potentially some common setup, like I have at the beginning of the setupTestCase() method, setupBasicPrerequisites() and worker initialization.
  • Note the use of SysTestCaseAutomaticNumberSequences attribute on the class. This allows not to worry about configuring them manually, just let the framework do it for you.
  • Note that my test will not use an existing partition and company. It should generally work in USMF as well though, and it's up to you to decide how much data setup you want to rely on.

OK, that's it for this post. It's a bit of a brain dump, but please read through the points, and don't hesitate to reach out with questions.

Hope this helps

3 comments:

  1. Quite interesting in this article is a transition from table-based approach to entity-based.
    It makes to take a look on dax elaboration process as on more abstract and lets operate by business-sounded entities unlike low-level programming. Since dax architecture implies necessity to do all modifications, concerned data structure changes, with the table (database specialists area) this blurs borders with just only business logic development (would require less specialized knowledge). As a result - elaboration time raises and raises significantly.

    Probably, dividing elaboration process in dax into 2 layers (not "layer" in ax2012 sence) with such hierarchy as: "low level" - as dax has at the moment and "declarative level" (as shown in the article for testing purposes only) would be extend the potential use of dax because of becoming faster for customization (in case of use existing master-data specification) and decrease of ownership costs.

    ReplyDelete
  2. Hi Vanya. Thanks for sharing this article. I tested and it works for the latest PU. However, I have several questions(and these questions are common from the partners when discussing the testing approach and should the auto-testing be used or not):
    1.
    You provided some example test that creates a WorkOrder and then checks that Workorder is created.
    Why you are checking only several fields for this. For example, WHSWorkLine and WHSWorkTable tables have more than 40 fields each, and you cover(check) in your test values only for several fields. This code is great for checking that you didn’t break the main function(that is great), but what about other fields?
    2.
    Partners need more example for this functionality usage. Is it possible that you provide real tests for some real issues (I just took several bugs from the latest PU Warehouse and Transportation Management)
    533573 Creating a sales order’s work without a pick location is not updating the on-hand figures correctly.
    535311 Cannot do material consumption from mobile device for shelf life items
    536899 The container contents report does not show an error if shipment does not have a dropoff address specified
    537772 Cluster profile could not be found please check your configuration Error is blocking Purchase Order Receiving
    If you share them in a new blog post as examples that will be a great discussion point
    3.
    Need more success stories for these tests. I am asking for this in almost every presentation on testing(including RSAT) but still didn’t get the answer. Can you maybe share some stories(with code) where a developer created and manually tested some new functionality, it worked on his machine and then he ran tests and found that his new code introduced some regression. What is a concern point here for the client – what is the price of finding one issue with auto-testing
    Basically answering all these questions(maybe in a series of blog posts) can make D365FO testing more popular

    ReplyDelete
    Replies
    1. If you read more about general testing principles, you'll notice that it is typically recommended, that for a single test, you should make it test only ONE thing. (https://stackoverflow.com/questions/61400/what-makes-a-good-unit-test)

      Now, as these aren't really unit tests, we are also testing more than one thing, also because we're not mocking away the data layer, as you can see.

      So we typically recommend that you validate things that you care about in the particular test. In our case, it was that you have exactly 2 lines, both are closed, and have the right quantity and locations.


      Success stories - yeah, that is a good idea. I'll see if I can pass it on to whoever does PR for this (if there's anyone).

      I'll look into #2

      Delete

Please don't forget to leave your contact details if you expect a reply from me. Thank you