https://www.infoq.com/articles/thoughts-on-test-automation-in-agile
Start Small
Balance the cost vs. ROI well and come up with a bare minimum working solution at the start.
Test Automation Backlog
A test automation backlog could contain a prioritized list of items such as:
- Parameterize the test environment for test execution.
- Integrate with Continuous Integration.
- Enhance reporting mechanism.
- Provide an option to attach error logs in notification emails.
- Collect performance metrics for workflow scenarios.
- Add tests to check for concurrent execution of critical test cases.
Tools Are Just the Means And Not The End
If you focus on the big picture, the real goal is to support new development efforts by providing rapid feedback to the team.
It is also important to keep your tests and test data independent of the selected test automation tool as much as possible.
Create Meaningful Tests and Don’t Try to Automate Everything
The most important part of your test solution is the “tests”.
Get It Out Of Your Local Machine
Execution Time Matters
Keep It Green
Precise Reporting
Make It Visible To Everyone
Test Early
Minimum regression sanity test as a start.
Involve users at the beginning.
Test Often
Corresponding scope of regression set up for each updated build.
Test Automatically
Build up automated SIT with auto-select production data/cases.
Automated meaningful test instead of everything. Note: we need manual effort for automated test building up.
To apply 80/20 rule.