[Development tutorial] Thoughts on test automation + more sample tests using ATL

I recently had an interesting discussion around testing and specifically Microsoft testing efforts and frameworks, namely, ATL. If you are not familiar with this framework - you've been living under a rock. Get out from under there and go read some articles about it, like the one below

https://kashperuk.blogspot.com/2019/05/development-tutorial-sample-test-tips.html

The discussion revolved around the complexity and costs of adding and maintaining tests. The partner also questioned how often we add tests, and if we can share more samples of such. You can also see his comment with a more concrete ask in the blog post above. 

Here's a partial quote:

Partners need more example for this functionality usage. Is it possible that you provide real tests for some real issues (I just took several bugs from the latest PU Warehouse and Transportation Management)

533573 Creating a sales order’s work without a pick location is not updating the on-hand figures correctly.

535311 Cannot do material consumption from mobile device for shelf life items

536899 The container contents report does not show an error if shipment does not have a dropoff address specified

537772 Cluster profile could not be found please check your configuration Error is blocking Purchase Order Receiving

If you share them in a new blog post as examples that will be a great discussion point

So in this post, I'd like to share 2 things:

1. Generally speaking, we at Microsoft, or rather, more specifically, we in the Warehouse management team, add one or more tests for every single customer reported issue. 

This obviously adds a cost to each bug fix we do, but with this we get a much more reliable release schedule and on-going confidence in our product offering, and have very few regressions despite heavy code churn.

The point I want to make here is - if you are an ISV, and you have not covered your solution with test automation, you should do it now, and reap the benefits. 

If you are a partner getting paid by the hour doing a specific customer implementation, it gets more tricky. However, I believe (I know) it is in your and your customer's interest to justify the cost of adding test automation.

Very frequently today, even with some of our larger customers, we hear test cycles of 1-2 months before taking a newer standard release. That puts a significant on-going cost on the customer, as well as stress, as we at Microsoft keep pushing everyone to be current. This could have been mostly alleviated with test automation

ATL is a framework which allows to write robust (and reasonably fast) test automation with a low cost investment, all things considered. 

2. We do not ship our test automation library. The code is not written with shipping in mind, so it's not necessarily "Microsoft production quality". The value for customers is also marginal. 

But, to answer the ask from the quote above directly, here are the tests added for the 4 issues listed. I did not include the class/test setup logic, nor did I verify these tests are able to run on your specific dataset (internally we have tests that are both dataset dependent and independent). So take them for what they are - mere samples that prove the point - all of these are rather easy to understand when reading, and are rather easy to write, if you know the scenario you want to automate. For all or most of the standard functionality there are already ATL "wrappers" ready to be used.

• 533573 Creating a sales order’s work without a pick location is not updating the on-hand figures correctly.

	[SysTestCheckInTest]
    public void soRelease_StopWorkOnLocDirFailureFalse_onHandCorrectWorkCompleted()
    {
		// Given
        const InventQty QtyOnReceiptLocation = 100;
        const InventQty SalesQuantity = 1;

        ttsbegin;

        var locDirFailure = whs.locationDirectiveFailures().sales();
        locDirFailure.LocDirFailWork = NoYes::No;
        locDirFailure.update();

        var locProfile = whs.locationProfiles().receipt();
        locProfile.setLPControlled(false).save();

        var receiptLocation = whs.locations(warehouse).create(locProfile);

        // Delete existing sales pick location directive to ensure no impact to the one created.
        var locDir = AtlEntityWHSLocationDirective::find('24 SO Pick', WHSWorkType::Pick, WHSWorkTransType::Sales, warehouse);
        locDir.delete();

        whs.locationDirectives().salesPick().setLocationRange(pickLocation).moveToTop();

        onhand.adjust().forItem(item).forInventDims([receiptLocation]).addOnHandQty(QtyOnReceiptLocation);

        salesOrder.addLine().setItem(item).setInventDims([warehouse]).setQuantity(SalesQuantity).setAutoReservation().save();
        ttscommit;

		// When
        salesOrder.releaseToWarehouse();

		// Then
        var work = salesOrder.work().singleEntity();

        work.lines().withTypePick().withLocationId('').assertSingle();
		
        invent.trans().query().forItem(item).forReferenceCategory(InventTransType::WHSWork)
			.forReferenceId(work.parmWorkId()).assertCount(0);

		onHand.assertExpectedOnHand(
            onHand.spec().forItem(item).withInventDims([warehouse]).withAvailTotal(QtyOnReceiptLocation - SalesQuantity));

        var soPicking = rf.login().salesOrderPicking()
			.setWork(work)
            .setLocation(receiptLocation);

        invent.trans().query().forItem(item).forReferenceCategory(InventTransType::WHSWork)
            .forReferenceId(work.parmWorkId()).withStatusIssue(StatusIssue::ReservPhysical).assertSingle();
        invent.trans().query().forItem(item).forReferenceCategory(InventTransType::WHSWork)
            .forReferenceId(work.parmWorkId()).withStatusReceipt(StatusReceipt::Ordered).assertSingle();

        soPicking
			.confirmPick()
			.confirmPut()
			.assertWorkCompleted();
    }  
  

• 535311 Cannot do material consumption from mobile device for shelf life items

      [SysTestCheckInTest,
        SysTestCaseAutomaticNumberSequences,
        SysTestFeatureDependency(classStr(WHSProdMaterialConsumptionJournalBOMBatchExpiryDateFeature), true)]
    public void materialConsumption_BatchIsExpiredBySchedDate_JournalLinesCreated()
    {
        const Qty BomLineQty = 1;
        const Qty ProductionOrderQty = 1;

        ttsbegin;

        // Given

        InventModelGroup fefoGroup = invent.modelGroups().fefoWithExpiryCriteriaBuilder().save().record();
        fefoGroup.PickingListBatchExpirationDateValidationRule = WHSPickingListBatchExpirationDateValidation::ReservationDate;
        fefoGroup.update();

        InventTable rawMaterial = items.whsBatchAboveBuilder()
            .setItemModelGroup(fefoGroup)
            .setPdsShelfLife(PdsShelfLife)
            .create();

        InventBatch validBatch = invent.batches().create(rawMaterial);
        validBatch.setExpiryDate(DateTimeUtil::getToday(DateTimeUtil::getUserPreferredTimeZone()) + PdsShelfLife).save();

        onhand.adjust().forItem(rawMaterial).forInventDims([floor, lp1, validBatch]).addOnHandQty(RMItemQty);

        InventTable finishedGood = items.whsBatchBelowBuilder()
            .setItemModelGroup(fefoGroup)
            .setPdsShelfLife(PdsShelfLife)
            .create();

        AtlEntityBillOfMaterials billOfMaterials = this.createBOMAndBOMVersion(rawMaterial, BomLineQty, finishedGood);
        AtlEntityProductionOrder productionOrder = this.createProductionOrder(finishedGood, billOfMaterials, ProdReservation::None, ProductionOrderQty);
        productionOrder.estimate().execute();
        
        productionOrder.scheduleOperation().setSchedDirection(ProdSchedDirection::ForwardFromSchedDate).setFiniteCapacity(false)
            .setFiniteMaterial(false).setSchedDate(DateTimeUtil::getToday(DateTimeUtil::getUserPreferredTimeZone()) + PdsShelfLife + PdsShelfLife).execute();

        productionOrder.start().setAutoBOMConsumption(BOMAutoConsump::Never)
            .setPostPickingList(false).execute();

        ttscommit;

        AtlWhsWorkExecuteProdMaterialConsumption materialConsumption = this.createMaterialConsumptionWithBatch(productionOrder, floor, rawMaterial, validBatch);
        
        materialConsumption.setQuantityToConsume(ProductionOrderQty);

        // When and Then
        materialConsumption.assertMessageOnClickOk("@WAX:MaterialConsumptionJournalLineCreated");

        materialConsumption.assertMessageOnDone(
            strFmt(
                "@WAX:MaterialConsumptionJournalPosted",
                productionOrder.pickingLists().withJournalType(ProdJournalType::Picklist).singleEntity().parmJournalId()));
    }
  

• 536899 The container contents report does not show an error if shipment does not have a dropoff address specified (Not using ATL, as this was an old existing test. More than 1 test was added / modified here)

      [SysTestCheckInTest]
    public void preRunValidate()
    {
        // arrange
        InventSiteLogisticsLocation inventSiteLogisticsLocation;
        delete_from inventSiteLogisticsLocation;

        Args args = new Args(formStr(WHSContainerTable));
        FormRun formRun = classfactory.formRunClass(args);
        formRun.init();

        FormDataSource containerTableDS = formRun.dataSource(tableStr(WHSContainerTable));
        containerTableDS.executeQuery();
        containerTableDS.markRecord(containerTable, true);

        args = new Args();
        args.caller(formRun);
        args.record(containerTable);

        WHSContainerContentsControllerTestable controller = new WHSContainerContentsControllerTestable();
        controller.parmArgs(args);

        Query query = new Query(queryStr(WHSContainerContents));
        controller.setRanges(query);

        Map queryContract = new Map(Types::String,Types::Class);
        queryContract.insert('Query',query);

        SrsReportDataContract srsReportDataContract = new SrsReportDataContract();
        srsReportDataContract.parmQueryContracts(queryContract);
        controller.parmReportContract(srsReportDataContract);

        // act
        container validateResults = controller.preRunValidate();

        // assert
        this.assertEquals(validateResults, [SrsReportPreRunState::Error, "@WAX:WHSContainerContentsReportMissingPrimaryAddress"], 'Unexpected value in validateResults.');
    }
  

• 537772 Cluster profile could not be found please check your configuration Error is blocking Purchase Order Receiving

      [SysTestCheckInTest]
    public void poReceive_QualityOrderAndAssignClusterEnabled_WorkAssignedToCluster()
    {
        const Qty PurchOrderQuantity = 2;
        ttsbegin;
        // Given
        whs.warehouses().ensureQualityManagementEnabled(warehouse);
        invent.qualityOrders().ensureCanCreate();
        whs.locationDirectives().qualityItemSampling(whs.locations(warehouse).stage().wMSLocationId, warehouse);
        whs.workTemplates().qualityItemSampling();
        invent.qualityAssociations().defaultBuilder()
            .setOrderType(InventTestReferenceType::Purch)
            .setApplicableWarehouseType(WHSApplicableWarehouseType::QualityManagementOnlyEnabled)
            .setItemRelation(whsItem.ItemId)
            .setSiteId(warehouse.InventSiteId)
            .setDocumentType(InventTestDocumentStatus::Registration)
            .setShowInfo(NoYes::Yes)
            .setAcceptableQualityLevel(100)
            .setQualityProcessingPolicy(WHSQualityProcessingPolicy::CreateQualityOrder)
            .setItemSampling(invent.itemSamplings().onePiece())
            .getResult();
        clusterProfile = whs.clusterProfiles()
            .putawayClusterManualAssignAtReceipt(WHSWorkTransType::Purch, whs.workTemplates().purchaseReceipt().parmWorkTemplateCode());
        ttscommit;
            
        this.addPurchaseOrderLine(whsItem, PurchOrderQuantity);

        // When
        var poItemRecv = rf.login().purchaseOrderItemReceiving()
            .setPurchaseOrder(purchOrder)
            .setItem(whsItem)
            .setQuantity(PurchOrderQuantity);

        AtlQueryWHSWork workQuery = purchOrder.work().withClusterProfileId(clusterProfile.parmProfileId());
                
        // Then
        poItemRecv.assertErrorOnConfirmUnitSelectionWithWorkQuery(workQuery, clusterProfile.parmProfileId(), 'Wrong message');
        invent.qualityOrders().query().forPurchaseOrder(purchOrder.record()).assertCount(1, 'Should have 1 quality order');
    }
  

Hope this helps. Do reach out to us with feedback / suggestions.

Do make use of CDE (Community Driven Engineering) to submit additions / bug fixes. We do expect ATL tests with those fixes.

Flights vs Feature management - practical overview

Terminology

The primary purpose of flights and features both is to implement a controlled roll-out of new functionality into the product, so as not to disrupt existing business operations. One of Microsoft's primary goals with OneVersion is to ensure we do not break anyone.

By their nature, both the flights and the features are transient in nature, i.e., they will be removed from the product after some time, leaving the new functional behavior as default going forward.

Let's now break it down a bit more, looking closer at what a flight is, and what a feature is, and who has control over them, and when they are enabled/disabled.

• A flight is a switch, which controls a certain, usually small, piece of business logic in the product, leading it one way or the other. Microsoft is in control of the state of the flight for customers in PROD. The decision to enable or disable a flight, as long as that does not impact the functionality of the environment, is not initiated by the company in most cases. Flights are, generally speaking, always OFF by default.
• A toggle is a business logic / X++ construct, that is typically based on flights. Similar to flights, it is either On or Off. So what's interesting here is what the default out of the box state of the toggle is.
• An "enabled by default" toggle, or, as it's more commonly known, a Kill Switch, is used for smaller changes, and the product behavior is changed right away, with no additional action from the users. Kill switches is a way for Microsoft to introduce bug fixes, yet safeguard against potential functional regressions, and as such, ensure we do not break anyone. If controlled by a flight, enabling the flight would turn the toggle and the new behavior off. Example: WHSShipmentConsolidationLoadLineLoadIdToggle
• A kill switch will get removed after around a year from the moment of introduction, making the new behavior apply to everyone always from then on. 
• A "disabled by default" toggle is used much less frequently
• To enable certain troubleshooting logic that is performance intensive and thus undesirable in PROD. Example: WHSInventOnHandForLevelCalculatorMissingSiteIdLogToggle
• To hide incomplete or work-in-progress features, so they cannot be enabled in PROD. Examle: WHSDockInventoryManagementFeature
• Enabling the flight would mean enabling the toggle
• A feature is, in a similar way, a toggle, which controls a usually larger piece of business logic, and is typically controlled by the superusers in the company. For this, they use the Feature management dashboard. Example: WHSWaveLabelPrintingFeature
• Features go through a life cycle, where they are
• In Development
• Not available for use by end users in PROD
• Private preview
• Available for use by end users, including in PROD, but only based on prior agreement with Microsoft
• Linked to a flight, which Microsoft needs to enable for the feature to appear
• Public preview
• Available for use by end users in PROD, based on decision to enable and configure the feature by superusers.
• Generally available / Released
• This is now the default behavior of the system, everyone has it. Depending on the feature, can still be configured "off".
• There are corresponding kill switches in place to disable new feature behavior if a severe issue is discovered with the feature. Only reserved for very rare cases.
• You can read more about Feature management here.

In an ideal world, partners/superusers should not need to know/worry about flights, and should control the behavior of the system solely through the Feature management dashboard. 

All of the above applies to PROD environments. The story with DEV/UAT environments is a bit different today for flights, where the partner/superuser has much more control. 

Note. Self-Service Deployment environments are controlled a bit differently, where above is also true for UAT type environments.

DEV / UAT environments

You can control the state of flights through the SysFlighting table. This is sometimes referred to as "static flighting". 

It's nicely described in an old post I found here, so I'll just paste the one line SQL statement you can use here instead:

INSERT INTO dbo.SYSFLIGHTING(FLIGHTNAME, ENABLED, FLIGHTSERVICEID) VALUES ('', 1, 12719367)

It's really as simple as that, but remember to evaluate, if the toggle is enabled by default or disabled by default before you start adding flights there. Maybe the logic is already doing what you want.

Tip For Inventory, Warehouse and Transportation toggles it is very easy, as you can see it from the name of the class the toggle extends from:

• WHSEnabledByDefaultToggle
• WHSDisabledByDefaultToggle

FAQ / Typical scenarios

I enabled the flight in UAT/GOLD by inserting into SysFlighting, but when going live, the flight in PROD is not enabled. Microsoft, please insert the record for me.

Flights in PROD are controlled in a completely different way, not through the SysFlighting table, and are also not enabled on customer's request, but only when it makes sense after evaluation by Microsoft.

Important. You should not expect that just because you turned on a flight in UAT/DEV, and liked what you saw, you'll get the same in PROD. Microsoft can reject your request for enabling particular flighted (private preview) behavior in your PROD environment.

If you discovered what you believe to be a functional or performance regression, and you managed to link it to a particular kill switch, please log a support request with detailed repro steps, so Microsoft can evaluate and fix. We will then typically enable the corresponding flight to mitigate short term.

I'm trying to disable a flight but it's not working. I inserted a row in SysFlighting and marked it as Enabled=0, but system still behaves in the same way as before

Flights are, generally speaking, only Enabled. So it just really depends on what kind of toggle this is in X++. If this is a "kill switch", enabling the flight will revert the behavior to how it was before the changes. If this is a private preview feature, enabling the flight will enable the new behavior (or simply show the feature in the Feature management dashboard)

This is easy to understand if you look at the actual implementation in the WHSEnabledByDefaultToggle class.

public boolean isEnabled()
{
    ...         
    if (flightName == carbonFlightName)
    {
        return !isFlightEnabled(flightName);
    }
    ...
}

This is basically reversing the condition for the flight. Meaning, enabling the flight will disable the toggle.

I enabled a feature, but I did not mean to, and now I cannot disable it

Some features are indeed designed in a way, where it is not allowed to disable them. The common reason here is that as part of enabling this feature, or as a result of using the feature for a short time, data was modified, which makes it difficult to revert to the old behavior. 

Also, not all features have a separate dedicated configuration switch in the product, so after you enable the feature, the product will in most cases start behaving in a different way.

So please carefully read the description of the feature, as well as evaluate the consequences of enabling it in UAT, before enabling it in PROD.

Microsoft enabled a private preview feature flight for me, but I still cannot see the feature in Feature management dashboard

Make sure you have clicked on "Check for updates" in the dashboard after 10-15 minutes after the flight was enabled.

Tip Microsoft also has a way of checking if the flight / feature is evaluated as On or Off for a specific environment, which can help you troubleshoot any issues, if it comes to that.

What is the difference between a Feature and regular Module parameters? For example, a Feature in HR - "Filter active positions" (This feature enables a position list filtered to only active positions). Why it is not an HR module parameter?

A feature is meant for controlling the roll-out of functionality, while a configuration parameter is meant for allowing the user flexibility in setting up their system. 

In the specific example above the intention with the "Filter active positions" feature is to expose it to all users (it's available in Feature management dashboard for everyone to see and enable), but in a manner which would not introduce new menu items / forms until this explicit decision is taken by the superuser, so as to ensure any documentation and internal task guides are properly updated beforehand. It is NOT the intention to make this behavior configurable going forward, and all companies will have access to the relevant menu item eventually (note, how the new FeatureClass property is set on it accordingly), while the feature class will be removed from the product. This approach also allows Microsoft to simplify the product, avoiding addition of unnecessary parameters.

That is not to say that both cannot co-exist. A module parameter or even whole new configuration screens can be added for proper setup of a new feature. They will only be exposed to the end users once the feature is enabled in Feature management. But then for the feature to function according to business expectations one or more settings need to be configured. An example of this is WHSShipConsolidationPolicyFeature feature, which you need to configure after enabling by setting up Shipment consolidation policies in the corresponding form.

More questions?

Please leave comments below, and I'll try to clarify!

[Development tutorial] Sample test + tips for using ATL (Acceptance Test Library) to implement tests in Warehouse management

Hopefully, by now everyone is familiar with the fact that we shipped our internal test framework for all ISVs, partners and customers to use.
We have published pretty detailed documentation about the framework on docs.microsoft.com:

https://docs.microsoft.com/en-us/dynamics365/unified-operations/dev-itpro/perf-test/acceptance-test-library

I, however, learn much better when looking at samples. We have shipped a couple sample tests in each area with the 10.0.3 release.
In this post, I would like to share another sample test case and walk through the different types of commands possible in ATL, focusing also on the part that drives mobile device flow operations.

Important note

ATL is not meant as a replacement for good quality production code, and should not be used to execute business logic in PROD environments

Sample test

This test will validate a very simple inbound flow where I want to validate I can confirm the putaway location by the location check digit:

• Create a new purchase order for a WHS-enabled item to a WHS-enabled warehouse
• Perform the registration of the item through the mobile device
• Perform the putaway through the mobile device
• Confirm the location confirmation is shown and is asking for the check digit
• Enter the check digit of the putaway location instead of the location ID
• Complete the putaway
• Validate the work lines are updated to Closed (not really the point of the check digit test, but just to show the capability)

[SysTestMethod]
public void confirmLocationWithCheckDigit_SimplePurchOrderOneItem_Success()
{
 ttsbegin;

 //========Given========

 purchaseOrder.addLine().setItem(item).setInventDims([warehouse]).setQuantity(PurchaseQuantity).save();

 rf.login().purchaseOrderItemReceiving().setPurchaseOrder(purchaseOrder).setItem(item).setQuantity(PurchaseQuantity).confirmUnitSelection();

 var work = purchaseOrder.work().singleEntity();

 //========When========

 var poPutaway = rf.login().purchaseOrderPutaway().setWork(work).confirmPick();

 poPutaway.parmCheckDigitControl().assertIsEnabled(); // Will automatically happen within set method below
 poPutaway.setCheckDigit(checkDigit); // This will confirm the Put to this location

 //========Then========

 poPutaway.assertWorkCompleted();

 work.reread();
 this.assertEquals(WHSWorkStatus::Closed, work.parmWorkStatus(), 'Work should be closed');

 work.lines().assertExpectedLines(
  whs.workLines().spec().withLineNum(1).withItem(item).withStatusClosed().withQuantity(PurchaseQuantity).withLocation(whs.locations().receipt()),
  whs.workLines().spec().withLineNum(2).withItem(item).withStatusClosed().withQuantity(PurchaseQuantity).withLocation(putawayLocation));

 ttsabort;
}

Looks pretty simple, doesn't it? And does not really require comments, as the code is pretty self explanatory for the most part, "even a functional consultant can read it" :)
A number of things I would like to call out here:

• Notice the name of the test method. We always try to name the method according to the Given/When/Then pattern (also explicitly called out in the method body), more specifically, When_Given_Then(). Or, for those more familiar with AAA, Arrange_Act_Assert()
• Note the method name does not start with test. You can still do that, but we prefer being explicit through decorating the method with the SysTestMethod attribute
• I skipped the test setup part - I'll post that below. Normally that would include steps generic for all tests, while the Given part in here would only contains a few things special to this particular test. In this test, we 
• create a purchase order line for a small quantity of a WHS-enabled item
• do the PO receipt through the mobile device, to create the putaway work.
• There is A LOT of defaulting that happens in these fluent calls. When writing code, you need to pay attention and override any default value. For example, 
• When setting inventory dimensions for the PO line, I only specify the warehouse, but that in turn sets the Site as part of the business logic. The Inventory status is also defaulted based on the Warehouse parameters. 
• When logging it to the mobile device in rf.login(), the default worker is used. However you can overwrite that and specify another worker to log in with. 
• The WMA flows should look like you'd have it on the actual device, with scanner actually sending a caret return aka clicking OK after entry. Note, that defaulting is also supported here, where we just proceed without explicitly specifying a unit
• Here, however, I used a dedicated method, confirmUnitSelection(), to improve readability. Inside, in the implementation, it's a simple OK button click. 
• It is possible to override this behavior, by disabling the automatic submission on the flow class. This way, you'll have the ability to for example set multiple controls at the same time before OK'ing, etc. 
• ATL allows for most areas to rely on either a special entity class, or the underlying record. Generally speaking, we always prefer to use the entity over the record. For example, take a look at how I retrieve the work entity for the putaway work created after registration.
• Process Guide flows are tested with a different set of classes at this point. We'd be interested to hear your feedback on, which approach is more convenient. 
• When doing putaway, I also use a "dummy" method, confirmPick() to confirm the pick up of the LP from the RECV location. These methods, however, could be doing some additional validation
• We have, for example, implemented some like confirmPutToBaydoor(), which will confirm we are in fact in a Put step, and the Location control has a value where it is of type "Final shipping location". Think through these scenarios when extending ATL
• We can also access the controls shown on the current screen right now, to perform additional validation, for example. I showcase that with the CheckDigit control, simply checking that it is enabled for data entry.
• This already happens automatically in the next call to setCheckDigit(), so you should consider where you really need this.
• assertIsEnabled() will also ensure the control is in fact visible.
• Lastly, I wanted to show how specs are used in a real example, where we in assertExpectedLines() specify the 2 lines we expect to see on the putaway work after execution of the putaway. This allows for really flexible validation of the data, that also is easy to read and understand.
• Check out how the flow class shows up in the debugger - it displays the XML with the controls, which really helps understand where you are in the flow if something in the test goes very wrong!

Entire test class

Below is the full listing of the xpp class with the test, including variable declarations and setup.

You can also download it from my OneDrive

[SysTestGranularity(SysTestGranularity::Integration),
SysTestCaseAutomaticNumberSequences]
public final class DEV_PurchaseOrderReceivingAndPutawayScenarioTests extends SysTestCase
{
    AtlDataRootNode                     data;
    AtlDataInvent                       invent;
    AtlDataInventOnHand                 onHand;
    AtlDataWHS                          whs;
    AtlDataWHSMobilePortal              rf;
    AtlDataPurch                        purch;

    InventTable                         item;
    InventLocation                      warehouse;
    WMSLocation                         putawayLocation;
    AtlEntityPurchaseOrder              purchaseOrder;

    private const WMSCheckText          CheckDigit = '12345';
    private const PurchQty              PurchaseQuantity = 1;

    public void setUpTestCase()
    {
        super();

        data = AtlDataRootNode::construct();
        invent = data.invent();
        onHand = data.invent().onHand();
        whs = data.whs();
        rf = whs.rf();
        purch = data.purch();

        ttsbegin;

        whs.helpers().setupBasicPrerequisites();
        AtlWHSWorkUserContext::currentWorkUser(whs.workUsers().default());

        item = invent.items().whs();
        warehouse = whs.warehouses().whs();
        whs.warehouses().ensureDefaultReceiptLocation(warehouse);

        putawayLocation = whs.locations().floor();
        putawayLocation.checkText = CheckDigit;
        putawayLocation.update();

        whs.locationDirectives().purchasePutaway();
        whs.workTemplates().purchaseReceipt();

        var poPutawayMenuItem = rf.menuItems().purchaseOrderPutaway();
        rf.menuItems().setupWorkConfirmation(poPutAwayMenuItem.MenuItemName, WHSWorkType::Put, NoYes::No, NoYes::Yes);

        rf.menus().default().addMenuItem(rf.menuItems().purchaseOrderItemReceiving())
                            .addMenuItem(poPutawayMenuItem);

        purchaseOrder = data.purch().purchaseOrders().createDefault();

        ttscommit;
    }

 [SysTestMethod]
 public void confirmLocationWithCheckDigit_SimplePurchOrderOneItem_Success()
 {
  ttsbegin;

  //========Given========

  purchaseOrder.addLine().setItem(item).setInventDims([warehouse]).setQuantity(PurchaseQuantity).save();

  rf.login().purchaseOrderItemReceiving().setPurchaseOrder(purchaseOrder).setItem(item).setQuantity(PurchaseQuantity).confirmUnitSelection();

  var work = purchaseOrder.work().singleEntity();

  //========When========

  var poPutaway = rf.login().purchaseOrderPutaway().setWork(work).confirmPick();

  poPutaway.parmCheckDigitControl().assertIsEnabled(); // Will automatically happen within set method below
  poPutaway.setCheckDigit(checkDigit); // This will confirm the Put to this location

  //========Then========

  poPutaway.assertWorkCompleted();

  work.reread();
  this.assertEquals(WHSWorkStatus::Closed, work.parmWorkStatus(), 'Work should be closed');

  work.lines().assertExpectedLines(
   whs.workLines().spec().withLineNum(1).withItem(item).withStatusClosed().withQuantity(PurchaseQuantity).withLocation(whs.locations().receipt()),
   whs.workLines().spec().withLineNum(2).withItem(item).withStatusClosed().withQuantity(PurchaseQuantity).withLocation(putawayLocation));

  ttsabort;
 }

}

A few extra notes:

• We recommend that you create a base test class for your test automation, that will hide the ATL navigation variable declaration and initialization, along with potentially some common setup, like I have at the beginning of the setupTestCase() method, setupBasicPrerequisites() and worker initialization.
• Note the use of SysTestCaseAutomaticNumberSequences attribute on the class. This allows not to worry about configuring them manually, just let the framework do it for you.
• Note that my test will not use an existing partition and company. It should generally work in USMF as well though, and it's up to you to decide how much data setup you want to rely on.

OK, that's it for this post. It's a bit of a brain dump, but please read through the points, and don't hesitate to reach out with questions.

Hope this helps

[Heads up] Explicit dependency required for enabling Process Guide based flows in Warehouse Mobile App

Introduction

With release of version 8.1.3 (build numbers going as 8.1.227.xxxx), we introduced an additional configuration option on the Mobile device menu items, which controls whether or not the system will attempt using the Process Guide framework for executing a mobile flow
This is a temporary configuration option, and is supposed to serve two purposes:

• Allow having a partially implemented Process Guide flow, where the customer will be able to decide if the scenarios the current implementation supports are sufficient to transition over, and get to experience the better performance, and be able to rely on the new flow with any tweaks and customizations necessary.
• Allow to run both Process Guide and WHSWorkExecuteDisplay version of the flow side by side, for testing purposes (and as a risk mitigation in case the new flow has a blocking bug)

There is one small BUT, however.

Detection

For any flows that were implemented before version 8.1.3, where the customer is already live and has multiple mobile device menu items created that are relying on Process Guide, those will stop working, with an error "Invalid Work Execute Mode".

Mitigation

The very simple mitigation for the issue is to simply go and enable the check-box to use process guide on the mobile device menu item. 

Mobile device menu items - Use process guide radio button

Note There are also a number of SYS flows that only have the Process Guide version where we did not want people to be able to use the legacy one - those are not editable on the form and will always use Process Guide

Impact

We expect that the impact of this should be very minimal, as there aren't that many Process Guide based flows live on 8.1.2 or below.
But if you do encounter this, please use the mitigation above.

Thanks

[Tutorial] Automating Release to Warehouse process through Wave Template configuration

Introduction

Currently many companies resort to manually choosing which shipments need to be added to a wave we are about to process, which is slow and error prone, as the user needs to check up on each of the shipments, understand where it is headed, what types of items are on it, which mode of delivery is used, etc.

Today I would like to show a demo example of how to configure wave templates, as to allow a more optimal distribution of orders, based on, say, delivery mode, or some other cut-off criteria. Automatically, without any user interaction.

Disclaimer

I will not be talking about Automatic release of sales/transfer orders to warehouse in this post. We'll discuss that at a later point in time. Today it is only about making sure certain orders are grouped on a single wave automatically, based on defined criteria, so that they can be processed in the warehouse at the same time. This of course comes hand in hand with automatic release, which would enable a pretty much automatic operation for certain orders

Wave template configuration

Imagine that I would like to accumulate and then process all 2-day shipping orders at once, every second day, so I have the picking done before 4 pm that day when the UPS (or other 3PL) truck arrives to pick them up.

For these orders I want to pick as fast as possible, and the inventory is typically already at the locations and does not need special packaging, so I will not include replenishment and containerization steps into it, and will configure my work template to take the goods straight to the bay doors. 

I would also like to limit the number of shipments on each of the waves to 10, so as to not overwhelm my workers. Instead, I'll release another wave later in the day if necessary.

Generally, I'd like the wave to be processed automatically as soon as 10 shipments have accumulated, but throughout the day my Wave planner might process another wave, depending on the situation.

Here's how my wave template would look for this case:

2 day shipping wave template configuration for warehouse 24

Here's how I have configured the corresponding query:

Note. No need to add any filters on Site or Warehouse, as these will be automatically added.

I want to only look at shipments for Sales orders, and only those that have a 2 DAY shipping mode of delivery. Pretty simple. If I wanted, I could make this much more granular, by adding additional table joins and adding filters on them.

2 day shipping wave template query

Sales orders

Now let's create a couple sales orders, set the Mode of delivery correctly, reserve the inventory and release the orders to the warehouse.

Note. The field that is transferred to the shipment is taken from one of two:

• Mode field on the sales order, if the Mode of delivery is not filled in.
• Mode set on the Carrier associated with the specified Mode of delivery.

Mode of delivery specified on Sales order 000910

I only added one simple line for the example, on both sales orders.

Sales line for Sales order 000910

Release to warehouse

When we release the first order to the warehouse, we will create a new shipment, which will have the correct Mode of delivery. We will then proceed to search for a wave template that matches it, based on Site, Warehouse, as well as the defined query. Our newly created wave template should succeed here.

We will not however be able to find an existing Wave in status Created, that would match the Site and Warehouse for the shipment that is also using the same wave template. Which means we will go and create a new Wave. 

Note. A new wave is created because of the Wave template setting Automate wave creation

When we release the second order to the warehouse though, we will find the wave created above, and because the conditions for adding to the same wave are met, and the wave template has Assign to open waves marked as Yes, we will add the shipment to the found wave instead of creating a new one.

Here's how the resulting wave looks:

Wave created based on the two sales orders

FAQ

Q: What if there is more than one wave in status Created, that was created based on the "right" wave template, and is for the same Site and Warehouse as the newly created shipment? Which wave will the shipment get assigned to?
A: The first of the waves that matches all of the criteria which also does not exceed the thresholds.

Q: Can I use fields directly from the sales order in the wave template query?
A: Yes, but you will need to join to get them. The right way would be to first join the load lines for the shipment, then the order lines related to the load lines, and then the sales order headers related to the sales lines. (Not directly from the shipment, even thought that will work just fine for certain businesses that always have only 1 shipment per 1 order)

Conclusion

As you can see, setting up a wave template that will group multiple shipments and create a new wave if necessary is relatively simple - you just need to get a few of the specifics correct in the setup, and then everything will work like a charm.

In one of the next posts we will combine this configuration with automatic release of sales orders, to achieve full automation of sales order release process, where no user interaction is necessary at all past the sales order creation and confirmation step.

[Development Tutorial] Tired of WHSWorkExecuteDisplay*? ProcessGuide to the rescue!

If you have ever tried extending the warehouse mobile flows in Dynamics 365 Finance and Operations, you know the code is very complex and extremely difficult to make sense of and modify without introducing regressions in unpredictable scenarios.

This was of course on our mind as well for some time, which is now I am pleased to announce the release of a new framework for implementing such flows, called ProcessGuide framework.

You can read the documentation we've published so far here:
https://docs.microsoft.com/en-us/dynamics365/unified-operations/supply-chain/warehousing/process-guide-framework

We will incrementally start converting the existing flows, and you should do the same with yours.
Give it a try and let us know if you have any feedback by leaving a comment here or reaching out directly!

Thanks

[Tutorial] Reading On-Hand information for Warehouse enabled items

Having difficulty understanding your own business data is probably not something you would expect to happen to you, but I know many people, including many 10+ AX experts who have a difficult time following the information about item availability when it comes to warehouse related items.

In this blog post, I will try to walk you through a specific example, line by line, column by column.

Note I am assuming that everyone is familiar with the general way we show on-hand information, specifically that you can summarize it by a different set of dimensions, viewing it aggregated by warehouse, location, or any other combination of dimensions.

What's the challenge?

The problems start when you need to account for the reservation hierarchy of the warehouse enabled item, and displaying how much is available at the different reservation hierarchy levels.

On-hand view, summarized by Site, Warehouse, Inventory status, Location and License plate

On-hand for item A0001, summarized by all storage dimensions

On-hand view, summarized by Site, Warehouse

On-hand for item A0001, summarized by Site and Warehouse

Step-by-step analysis of the data

1. If we look at the Physical Inventory column, the on-hand there is always at the lowest dimension level – meaning we only see values for rows where you have all of the dimensions specified, down to the License plate level.
1. That’s why in the first row, where there’s no location/license plate, the Physical inventory is shown as blank (0). Same for the 5^th row, where for location FL-001 with blank LP there’s nothing in physical inventory. (RED)
2. Note that for BULK-001 there is physical inventory, even though there is no LP – that’s because this location is not tracked at License Plate level, so the Location IS the lowest level (ORANGE)
3. For some of the lines shown, there simply isn’t on-hand, but we used to have on-hand on them, which is why they show up. (YELLOW)
4. The rest (GREEN) are quantities that are on license plates on corresponding locations. You can sum up and see the total quantity of this item we have in the warehouse is 205 (but, again, that’s not displayed in the first row, unless you change which dimensions are displayed – if you only display Site and Warehouse, for example, that’s what you would see – 205. See screenshot #2 above)
2. Reservations are done at different dimension levels. So, for example, Sales order reservations are at “Above-location” level, while Work reservations are at “Location” level. Reservations at other levels are also possible
1. That’s why in the first row, we have 30 in the Physical Reserved column – that is 3 different sales orders having a reservation against the quantity 10 each. They don’t care which location/license plate we take the items from, as long as we have enough on the warehouse/status level. (BLUE)
2. Work reservation are at the location level. That’s what we have in row 5, where 2 work orders each reserve 10 units, and those units are to be picked exactly from location FL-001, but we do not care which license plate they will be taken from. (DARK BLUE).
3. All the other rows are empty, as we have no reservation at the location+LP level here (Even though it is possible, generally speaking)
3. Available physical column is where it gets a bit more tricky to understand. Let’s take it bottom up, so it’s easier to understand
1. On third line, we have 100 physically there, and there are no reservations for any of it, so all of it is available. Thus, it’s 100
2. Same for the 4^th line, 10 there and all of it is available.
3. Now, important part here. On FL-001, LP 24, I have 90 physically, but 20 is reserved. So why the heck is 75 available, instead of 70? (PURPLE)

                                                    i.     It is due to the reservation being on a higher level. The reservation is done at the location level, so when calculating available, it accounts for other license plates in this location as well, LP 000USMF-000..55 has 5 on it.

                                                   ii.     Another way to think of it – how many items can I take away from this License plate, so my existing reservations are still respected. The answer is – 75, because you’d have 15 left on this LP and 5 more on the other LP within the same location. The warehouse-level reservation is also still respected, because we have enough in other locations.

4. On FL-001, 000USMF-00..55, we have 5 available. That’s what is physically there, and because we have enough in the other LP, all of this is available for taking for another purpose.
5. The total quantity available at the location level matches what I described above, and equals 75, as it has 95 physically on it on multiple LPs, and 20 is reserved for work.
6. On the warehouse level, we have 205 physically, and we need 30 for sales orders, so 175 is available to be reserved for other purposes. As you can see, that’s not a direct sum-up of the other rows. Same as the available quantity on FL-001 was not a sum of what is available at each LP. The calculation logic is the same here.

Additional information

If you are still curious for more after reading the below, check out this great 1 hour presentation my colleague Lennart did at a conference a while back, explaining the reservation engine and on-hand impact: https://www.youtube.com/watch?v=--_didmZKHo&t=4s

Telemetry as part of the Dynamics 365 Finance and Operations life-cycle

How much telemetry are we collecting?

A lot, like, really a lot!

That includes kernel level information, like an online user session requesting web access to a particular AOS to perform business operations through the web UI, a web service request to handle a mobile device operation, an OData request for exporting or importing data, exceptions and other infolog messages displayed to the user, etc.

It also includes specific application-level information, like details about each step during an inventory update, or information about the different wave processing steps, the specific flow happening on the warehouse mobile device, etc.

And we keep adding to it to have more and more granular information about what exactly is happening in the system at any particular point in time.

How can I access all this telemetry?

You can find it under "View Raw Logs" under Environment Monitoring in LCS.

It's very well described in the following two articles, so please read through those at this point:

• Monitoring and diagnostics tools in Lifecycle Services
• How to use Environment Monitoring View Raw Logs

You can then use the different search options described above to query out the specific events you are interested in.

What about Warehouse - specific telemetry, say, Wave processing?

Depending on the version of the product you are running, it differs slightly. 

Here's how you would search for wave processing events on releases before Fall release of 2018:

Raw logs search criteria

Note The above query options is a preview feature, so is most probably not visible to you yet.

If you then wanted to filter on a specific wave, for example, you could add that to the search criteria as well. 

Note that due to compliance, all of the information is not exposed directly, but rather RecIds are used. So you'd need to retrieve the RecId of the wave you wanted to investigate.

Here's some of the fields you should take note of:

• The TIMESTAMP column would tell you when exactly the event occurred
• RoleInstance would show, which AOS the activity happened on. This could be useful when troubleshooting caching and other cross-AOS issues
• ActivityId is a way for you to limit to only a specific smaller process (for example, a specific allocation thread), and dig deeper, for example, to analyze the slow queries that happened as part of that activity
•  infoMessage would contain information about the actual wave step performed, as well as specific details about the step, like how many load lines were processed, how long it took, etc.

TaskName: WhsPerformanceTaskStop, ruleName: waveProcessing, actionPerformed: runWaveStep, durationInMilliSeconds: 0, details: {"allocatedLoadLines":"500","custom":"no","waveId":"5637815827","waveStep":"WhsPostEngineBase.allocateWave"}

From Fall release onward

You could search for WHSPerformanceTaskStart/Stop directly in the TaskName instead of as part of infoMessage. The rest still applies.

What about mobile device telemetry - do we capture that?

Of course we do :)

Just search for TaskName == WhsUserActivityEvent. 

This event contains the same basic information as mentioned above, as well as stuff specific to the mobile flow:

• Company, Site and Warehouse, where the warehouse user is operating. (RecIds)
• WorkExecuteMode and step for the specific flow. You'll need to lookup the actual step in code
• GUID of the mobile device user session
• WorkTransType, as well as WorkTable and WorkLine RecIds being processed
• RequestXML - this is the actual request, except all the data and labels are scrubbed, so it shows just what controls are shown on the screen

This event, if you search for it in code, is invoked at the very end of processing the user input from the mobile app.

You can correlate these events with those of TaskName == RequestContext, where the url of the request contains "/api/services/WHSMobileAppServices/WHSMobileAppService/getNextFormHandHeld" - this is the actual web service call as it arrives at the AOS, so the point in time when the AOS starts handling the mobile app flow step. 

Everything that happens in between with the same ActivityID is part of the flow, whether that is slow queries, error messages or other relevant events.

I don't see these events - what should I do?

We have back-ported a lot of the telemetry through hotfixes. Search for it on LCS for your specific release. 

But I would also like to take this opportunity and move up to the latest release. There is soo much goodness in there!

Should partners be adding telemetry?

ABSOLUTELY!

• If you are a client, I suggest that you start insisting your partners adds telemetry with any new code the push into production.
• If you are a partner, I suggest you start adding it asap - you'll save yourself a lot of time going forward, if the customer has an issue with your code. Since you cannot just debug in production any longer, you'll need to rely much more on alternative ways of telling you what exactly happened. Telemetry is the solution.

Where do we start? Do we have some examples?

We are working on the guidance for this and will share this out very soon.

Yes, all of the application telemetry is added directly as part of the application code, so should not be too difficult to find.

I suggest that you rely on the same event "InfoLogMark" as shown below as well.

Consider adding a using statement to reduce the invocation.

Microsoft.Dynamics.ApplicationPlatform.XppServices.Instrumentation.XppRuntimeEventSource::EventWriteInfoLogMark(


                                      Exception::Info, strFmt('TaskName: , etc.', );

What else should be captured out of the box that is critical for you?

Let me know in the comments!




Thanks

[Learning] Transportation management in Dynamics 365 Finance & Operations

I don't talk much about TMS in this blog, so I decided to correct that a bit, and post a set of educational links for those interested in starting up with Transportation management in AX.

If you are already familiar with all the concepts, you probably won't gain that much from these, but for those just starting up, I think it can provide a great overview of what the system can do.

You can start by reviewing the presentation one of my colleagues did a while back.

For a more in depth look, you can go through the TMS learning course (note it is based on AX 2012 R3, but the functionality is largely still the same in the latest release):

https://mbspartner.microsoft.com/AX/CourseOverview/1123

A couple more learning videos are available through the below link:

https://mbspartner.microsoft.com/EOP/Topic/77


Transportation management implementation guide and white papers available can be found through our SCM team blog posts:

http://blogs.msdn.com/b/dynamicsaxscm/archive/2014/11/21/transportation-management-system-implementation-guide.aspx

https://blogs.msdn.microsoft.com/dynamicsaxscm/2015/01/27/new-white-papers-for-tms/

And, finally, we have a permanent page on doc.microsoft.com, that you can access through the below link. This is where most of the upcoming functional changes and updates will be communicated going forward.

https://docs.microsoft.com/en-us/dynamics365/unified-operations/supply-chain/transportation/transportation-management-overview


And, of course, if you have further questions after this, please reach out, so we can connect you with the right people to help.

Thanks