Project-Based On Machine learning with the Integration of Jenkins, Docker, RHEL 8, Github and Git

5 min readJul 11, 2020

In today’s scenario if we see anything around us everything is automated from mobile phone to spacecraft ,so basically we are in the world of Automation

so, the main question arises is:-🤔

What is Automation?

Automation, or Labor-saving technology is the technology by which a process or procedure is performed with minimal human assistance. Automation or automatic control is the use of various control systems for operating equipment such as machinery, processes in factories, boilers and heat treating ovens, switching on telephone networks, steering and stabilization of ships, aircraft and other applications and vehicles with minimal or reduced human intervention.

Automation covers applications ranging from a household thermostat controlling a boiler, to a large industrial control system with tens of thousands of input measurements and output control signals. In control complexity, it can range from simple on-off control to multi-variable high-level algorithms.

Why do we need Automation?

Automation is needed in Industries to increase the accuracy of the Machine Learning model. Instead of changing the model manually, we do with the Dynamically with the help of the Integration of Jenkins, Docker, RHEL 8, git, and machine learning model.

Advantages of Automation:-

Decrease the Time of the Developers and Operation guys — manually uploading, changing the code.
Increases the Accuracy of the model, it automatically tweaks the various things in the model and increases the accuracy.
No Need of Monitoring the model by the persons manually because this is done through Jenkins’s job.

these are some of the advantages that i personally experienced….

So lets talk about our the task that i am going to perform…

Task Overview:

1. Create container image that’s has Python3 and Keras or numpy installed using dockerfile

2. When we launch this image, it should automatically starts train the model in the container.

3. Create a job chain of job1, job2, job3, job4 and job5 using build pipeline plugin in Jenkins

4. Job1 : Pull the Github repo automatically when some developers push repo to Github.

5. Job2 : By looking at the code or program file, Jenkins should automatically start the respective machine learning software installed interpreter install image container to deploy code and start training( eg. If code uses CNN, then Jenkins should start the container that has already installed all the softwares required for the cnn processing).

6. Job3 : Train your model and predict accuracy or metrics.

7. Job4 : if metrics accuracy is less than 80% , then tweak the machine learning model architecture.

8. Job5: Retrain the model or notify that the best model is being created

9. Create One extra job job6 for monitor : If container where app is running. fails due to any reason then this job should automatically start the container again from where the last trained model left.

Understand! good ,lets dive together in the world of Automation…….

Creating a Dockerfile: Installing all required modules so our Machine learning model can run in docker container

Build a docker image :

Creating Job 1: whenever a user push its code to github ,this job 1 will copy files from github…….

Creating Job 2 & 3 : this will launch a container , in which it will automatically start training the model

This job checks the pulled file content and searches which type of file is this means which model is created Ex- CNN, Sklearn, etc. If it is CNN then it launches the container from the image that we had created . After that, it links the data-set that is stored in Rhel8 and run the python file to trains the model.

Creating Job4 :Tweaking the model if accuracy is lower than desired accuracy

This job first checks the accuracy of file.py and if the accuracy is less than 90% it tweaks the code by adding the CRP (Convolution Relu Maxpooling) layers and when it is greater than 80%, it doesn’t tweak the code and send an email to notify that your model has achieved best accuracy.

Creating Job 5: Retrain the model

This job will the push the file.py to github if jenkins sees any commit in the file. Without this job automation cannot takes place.

This job will keep on monitoring the process, if due to some reason, the container failed to run, then this job run the container and trains the previous model, in case of failure it pushes the email.