I was joining a development team some many years ago. One of the tasks assigned to be in my main responsibilities was building product deployment module. The thing about deployment related activity is that it is not that fun comparing to coding. One of my colleague even expressed her thankfulness for me taking care the task since nobody really prefer to take it.

And it was very challenging indeed. It might be the fact that it was a new code base or I just didn’t have enough experience in this kind of tasks. It took me a lot of time and effort to get it done. I am not going to lie. There were a lot of painful moments and I started questioning on why it needed to be this hard. Or I am just not smart enough to do it right.

There were other issues with my life at the time. A lot of negativity thought floating around my head. When you are in difficult time, everything looks like punishment from universe with determined intention to make you and only you suffer. And sometimes I saw the task as one of the punishment.

A year later I joined another team. It was also a new code base and I also needed to get my hands on creating deployment script again. The strange thing was that I didn’t feel much pain this time. It was still annoying and taking much effort but I wouldn’t say it was a pain to carry it out. It might be about me gaining more experiences or it was just simply because I didn’t think of it as a pain despite the fact that the nature of the task was still the same.

I later moved to Japan last year to join a team which also starting a new code base. The team decided to use CircleCI instead of Jenkins for build and deployment pipeline. The down side is that we will not have the pre-developed scripts and modules support in our internal Jenkins build system team. We have to develop our own deployment module connecting to Kubernetes cluster. And just like that, I found myself writing another deployment module.

It is still annoying and take a lot of time. Kubernetes is flexible but also comes with the cost of complexity in putting many things together for it to work. Those YAML manifest looks quite intimidating. One of my colleague said that this thing is too complex.

Again, I don’t see the task as a pain. I still don’t enjoy it that much even though I know how much important it is to have a solid deployment module (Good product that can’t be deployed to production is not much useful ). The good things I have noticed during developing this module is that all the tricks I learned from the past is still very applicable here.

• It is still mostly declarative style. We declare things we want to put together and let the system wire them up. It is mostly about putting name/config/token in the right place with the right indent in the right sequence in the right file. A lot of bug is just because typo.
• There are hooks to add imperative script. Just like puppet script that is declarative but also able to call predefined script. We could also add some script e.g. docker container startup script to customized deployment behavior.
• All the good practices in deployment are still the same. After deploy, how do you know the service is up and running well. Does it have a health-check endpoint to verify. What should be do in the health-check function? Should we try calling some light-weight request to system dependencies?
• All those things that could go wrong in deployment are the same. The deployment go well in Dev and Staging environment but it failed on production since something like networking/proxy/Access control is different in production. Yeh, it could happen in every type of deployment system

My boss said he was quite impressed to see me able to get the deployment up and running in a pretty short time. Who knows that the task I was once considering as a punishment from universe will become the a highlight task for me recently.

This make me think about my difficult time many years ago. I saw only pain and negativity and all the thing I was doing is just a waste. It is like zooming in to look at one dot in a picture. Little I knew that one dot connecting with many other dots could create a meaningful picture.

If you are experiencing a tough time. Make the most out of the situation. Not all things you are enduring is wasteful. Something could be a stepping stone for the next great thing. The whole picture might reveal itself some years later.