QA Diary : Low Cost Powerfull Building Automation Infrastructure
Every day we need to run our all automation jobs in order to make sure the system has already stable. Currently, blibli has more than 200 Jobs that will run at the same time with more than 8.000 test case. it will make our automation is more challenging to run, so how we can stabilize that all-around 200 Jobs automation at the same time, the journey will start here.
The Beginning
in the beginning, we start with 3 Slaves and one slave is only able to handle 5 jobs at the same time.
So in total, we have 15 jobs that we can. Because it’s windows slave, so we don’t need to create another server or instance inorder to run UI Automation Test because if you are running automation in there ( Windows ) the browser will spawn automatically inside the slave so we don’t worry about that.
But the problem begin….
We have list Top of the problem That we got as long as we use windows slave
- Bluescreen
- BlueScreen
- BLUeScreen
- BlUeScreeen
Yeah, sometimes windows are crashes and make all automation that runs in the same environment is failed at the same time.
For example, there are 5 Jobs that run at Slave 1, then Something Bad happens, the Emoticon “ :( “ appear in that slave that makes all 5 jobs is failed at the same time. so we need to figure out how we can increase the capacity and have a low cost for that. And you are lucky, we have the Recipe that we use from 2018.
Quote From Windows :
Your PC ran into a problem that it couldn’t handle, and now it needs to restart.
Recipe
As we know you will be familiar with Jenkins, Bitbucket, Selenium, etc. Yeah, we are using mostly free open source software that supports our all jobs, we are not yet using cloud computing like GCP or AWS, or Azure, we still utilize the existing infrastructure that already exists in our Data Center.
The kind that you need is :
- Jenkins
Yeah, Open source Free CI/CD that you can customize. Actually, any CI / CD will Work so it depends on your company which CI/CD that you using as long as it’s able to do “Fancy Shell Scripting Programming” like Jenkins it’s able to work.
2. Docker builder
This docker builder has a feature for isolated automation run, so one container only handles one job. and it minimizes the impact on other jobs. For example, Jobs A, B, and C running in One Slave, then the docker builder will create Container to handle each jobs ( 1 container , 1 jobs) . so if jobs A make an error or crash , the jobs B and C will not be impacted. It will more be isolated. There is some way to integrate it between Jenkins and Docker, but in Blibli we are using this plugin
It’s easy to understand and fast to install and easy to scale up if in the future we need more slaves.
3. Selenoid and GoGrid Router
and for UI automation, we combine the tools between Selenoid and GGR, Selenoid itself is used for running browser automation based on container/docker based. it’s easy and very reliable to set up. Actually, if you are familiar with other grids for example Selenium Grid or other tools you can use that, but the reason why we are using GGR and Selenoid because the server is running on Go Language instead of Java and it’s faster than Java.
One instance only take +-100MB for the ram to run the entire browser automation test. And now we have 5 Selenoid with each of selenoid instance can handle 5 Browsers at the same time.
The other reason that we choose selenoid instead of other Grid because selenoid Support So many version of the browser, you can check the browser that supported by selenoid by visiting this URL
Browser Images — A set of browser images for Selenoid and Moon (aerokube.com)
The Infrastructure
After you get all the recipes, Now we will share the infrastructure diagram that we use.
From Jenkins, we Instruct docker slave to create a container in order to handle the jobs.
If it’s API testing, After the container is created, the automation can directly communicate with Testing Environment.
If it’s UI Automation, the automation will request the browser that is needed to GGR( for example Chrome Version XXXX) and then the GGR will allocate the Selenoid Instance to create a browser container based on the request from automation. And the last the selenoid will connect to the testing environment. And the automation will be running as usual.
Current status
We already have a Maximum of 25 Parallel execution in UI Automation,
if you compare it to sauce labs we can save up to 1599 USD per month only for Running UI Browser Automation and we can run as long as we need in that grid instead of limited X minutes time.
Closing
Currently We are open For SDET / QA Position , if you want to learn this more , you can contact me at argo.triwidodo@gdn-commerce.com or by telegram @argotriwidodo
And see you in Blibli 😉
