What Test Level Is Applied and Where When Working with Copado?

Updated 4 weeks ago by Copado Solutions

Overview

The test level in a pipeline connection is only applied when recalculating the pipeline on the Pipeline Manager page:

Pipeline Manager Page

The Promotion Test Level on the user story is applied when using the Promote & Deploy checkbox. The deployment step for that user story will be created with that value:

Promotion Test Level

The test level in the Deployment Details popup that appears when you click on Create Deployment in a promotion is applied to the Deployment record created from there:

If a deployment is created manually, or if you just need to modify the test level applied from the options above, you can do it from the Test Level field directly in the deployment step:

deployment step

The test level value selected in the deployment step will be overridden when deploying if the Run all local tests checkbox is enabled in the Environment record of the organization to which you are deploying:

Environment

Use Cases

In the scenarios  below you can find the different places where you can run Apex tests from as well as the test level that is applied in each case:

  • Scenario 1: In the user story you click on the Manage Apex Tests button. 
    • What happens: The following records are created: 
      • Promotion
      • Deployment (Check-Only)
        This is not visible in the UI. 
    • Where tests are executed: Source org
    • Scope of included tests: All Apex tests committed to the user story or included through the Add Apex Tests button.
    • How tests are executed: The coverage returned is based on the code of the classes in the source environment, not the feature branch. The results are attached to the user story: ApexTestResult.
    • Test level: Test level is applied in this order:
      • Specified Tests
      Manage Apex Tests button
  • Scenario 2: In the user story you click on the Validate Changes button.
    • What happens: The following records are created: 
      • Promotion
      • Deployment (Check-Only)
    • Where tests are executed: Destination org
    • Scope of included tests: All Apex tests committed to the user story or included through the Add Apex Tests button.
    • How tests are executed: The coverage returned is based on the code of the classes deployed to the destination environment and that code is taken from the promotion branch. The results are attached to the user story: ApexTestResult.
    • Test level: Test level is applied in this order unless you manually overwrite it:
      • User story Promotion Test Level field 
      • Automation test level 
      • The environment setting Run All Local Tests trumps all.
Validate Changes button

  • Scenario 3: You add a Git Promotion step to a deployment and select the test level.
    • What happens: The following records are created: 
      • Promotion
      • Deployment 
    • Where tests are executed: Destination org
    • Scope of included tests: It depends on the selected level. 
    • How tests are executed: The coverage returned is based on the code of the classes deployed to the destination environment and that code is taken from the promotion branch. The results are visible by examining Copado's mock-up of the Salesforce deployment status page knobs.
    • Test level: Test level is applied in this order: 
      • User Story with Promotion Test Level selected (default):
        • Run All Tests in Org
        • Run Local Tests 
        • Run Specified Tests - this will run all Apex tests that have been committed to a user story or added via the Add Apex Tests button on the user story. 
      • Promotion with Test Level selected (override): 
        • Run All Tests in Org
        • Run Local Tests
        • Run Specified Tests - this will run all apex tests that have been committed to a user story or added via the Add Apex Tests button on the user story. 
      • Automation test level
      • The environment setting Run All Local Tests trumps all. 
If the Copado Continuous Delivery automation invoked has one test level selected and either the user story or the promotion has a different test level selected, then the Copado Continuous Delivery automation Apex Test with Validation deployment step will execute one test level and the Git Promotion step will execute a different test level. Keep in mind that, if multiple user stories are included in a promotion, the promotion's test level overrides the promotion test level selections on the individual user stories. This is not a problem per se, but you will likely get different results on the two deployment steps.

Git Promotion

  • Scenario 4: You configure an Apex test with validation automation.
    • What happens: The following records are created: 
      • Promotion 
      • Deployment (Git promotion step with Check-Only)
    • Where tests are executed: In the org for which the automation is configured in the connection behavior.
    • Scope of included tests: All Apex classes and tests in the promotion branch
    • How tests are executed: The coverage returned is based on the code of the classes deployed to the destination environment and that code is taken from the promotion branch. The results are attached to the deployment step record page under Apex Test Results.
    • Test level: Test level is applied based on selected level:
      • Run All Tests in Org
      • Run Local Tests
      • Run Specified Tests - this will run all apex tests that have been committed to a user story or added via the Add Apex Tests button 
  • Scenario 5: You click on the Run All Tests button on the Credential record.
    • What happens: Copado executes all the Apex tests in the selected credential. 
    • Where tests are executed: The org for which the credential has been authenticated.
    • Scope of included tests: All local apex tests 
    • How tests are executed: The coverage returned is based on the code of the classes in the org for which the credential has been authenticated. The results are attached to the Apex Test Results on the Credentials page.
    • Test level: All local tests
Credential record

Additional Considerations

  • If you are working with Copado Continuous Delivery and the test level on the Apex test automation is different from that on the user story or the promotion, the Git Promotion step will execute one test level and the step resulting from the automation will execute a different test level.
  • If you are promoting multiple user stories in a single promotion, the promotion test level will override the test level specified on each individual user story.
  • The Run all local tests checkbox on the Environment record not only overrides the test level selected on individual user stories or promotions, if this checkbox is selected, Copado also runs all local tests in the destination environment of the deployment even if the promotion doesn’t contain Apex code.


How did we do?