One Month into working remotely — a review of what worked well and what did not
On Tuesday the 17th of March, almost all of us working at company’s Surry Hills head office waves each other bye as usual before the end of the day. Most believe we would probably meet again in the next week — a trial of having everyone working from home was starting the next day for the rest of the week. However the situation of COVID-19 spreading in Australia evolved quickly in this very week and in the weekend, this trial turned into an ongoing arrangement with no end day confirmed.
Fast forward, when I am writing this, it is more than one month since the beginning. The working from home arrangement is mostly a success: Business activities are going, teams are still functioning, developers are completing tickets at seemingly usual velocity and releases are flowing on. But of course, not everything is business as usual. With most of us giving up the hope that we can have everything back to what they were in a snap, I believe this is a good time to take a look at what worked and what did not from technical operations point of view. This way we could be better strategised in the long run.
The good
👍 AWS received almost no impact from the situation of virus.
We are proud to be digital native and cloud-first. While no one is unaffected by the pandemic, AWS included, their expertises and resources shielded the impact and we as a customer benefit. All of AWS’ services and tools are available to our developers at home the same way as they are in our offices. And all our digital assets are up and online with no degradation and interruption. In fact, the biggest players in public cloud — AWS, Google and Azure, all performed exceptionally well under the current situation. This solid foundation our products and business built on, allowed decision makers to focus more on the important things such as employee wellbeing and acted quickly on the shifting needs of our customers.
👍 Cloud-based services integrated in our workflow smoothes the transition in and out of the office.
All employees in the company are already exposed to the core collection of tools such as G Suite and Slack. The tools are a part of their existing workflow, sometimes the geographical distance of some mandates them to communicate over the internet too. When the pandemic hits, publicly available, cloud-based tools are very accommodating in terms of shifting workflows online. There is minimal training gap for our people to collaborate using those tools instead of face to face in the office. There are a few things we need to resolve here and there. For example finding a replacement of planning poker and retrospective boards — We are just obsessed with the physical feeling of those cards and notes, but it’s unusual time now. It definitely feels different cutting out the last bit of face to face talking and shift all conversations on Slack, but not needing to fight for a room to schedule meetings feels oddly satisfying at the same time. We can tell the usual collaboration among people continues, with the support of a great selection of tools.
👍 Authentication integration with Cloud services paid off.
We do still run an internal directory to support some of our services. But we have made the effort to integrate our directory with publicly available identity services such as G-Suite and Azure. Being able to single-sign on with corporate credentials on the public internet proved to be valuable for us. At this time when everyone is at home and we are pretty much all out of the trusted corporate network, this integration has provided a smooth experience. Instead of mandating everyone to connect VPN so they can reach our directory, we are able to use our corporate credentials on most of our tools over the home internet. One password to remember, no VPN required and no bandwidth bottleneck on top your home connection. Plus the login experience for the tools stays the same whether you are at home or in the office.
The not-so-good
👎 SMB file share is difficult to use over home connections.
Well sometimes you just have to admit that certain workflows just aren’t going to be smooth enough in a distributed environment. One of the teams in the business runs a workflow that deals with RAW video footages. They have a central storage in the office for their media and it runs SMB sharing. Everyone is quite happy with the GbE LAN link in the office, but performance and experience suffers big when trying to reach the storage from home for obvious bandwidth reason. The bandwidth of home connection is not the only issue here. Even if your are willing to open wallet wide and high-speed FTTP NBN is available at your home, it is still difficult to take advantage of your full home bandwidth with SMB share — they are rarely exposed on the public Internet, means your useable bandwidth to the share are shaped by VPN too. Compared to the scalable cloud options such as G Drive and MS OneDrive, SMB is really designed for LAN and runs best on a much smaller scale.
👎 VPN and traditional networking is a pain.
As much as I personally dislike VPN, it is still a widely used technology. The idea of “extending trusted corporate network to the untrusted public internet” is popular. But what comes with it is the big cost of complexity. By doing this you are overlaying a part of the network you usually have pretty good control of — consistent bandwidth, well designed route capacity and even good physical security, over another network we simply give up describing in detail and replaced with a simple cloud icon on most diagrams. When it comes to implementation, the devil in details may even confuse experienced engineers — try creating a site-to-site VPN link between devices from different vendors and you may have your first hand experience. There is one internal application that we finally made available to everyone working remotely one month into the journey. The organisational and technical effort we put into it is night and day difference compared to AWS, Slack, CRM and the many development tools we use which are available on the public internet.
The New Grand Plan
Many have claimed this global pandemic is going to leave long and profound changes in our lives. I tend to agree and we will see an increase of working from home arrangements even after the pandemic is over. Businesses are going to take this into account and change their workflow to accommodate. On the technical operations perspective, there should be a new thinking on how we approach enterprise systems. I am never satisfied with “works” — kicking the existing apps down the road by creating a virtual “internal” overlay is just clunky.
What should be the key attributes of an internal application optimised for both in and out of the office be? I believe it should be
- Secure. No compromise to the security of the app and data transmitted between remote employee and corporate.
- Fast. Take advantage of all available client bandwidth regardless of the type of connection.
- Convenient. Use your enterprise identity across multiple applications.
Zero Trust Security is a promising thinking about how to achieve all of the above. The fundamental idea of ditching the traditional security perimeter and deploying all applications on the internet is exciting. It resembles with our own experiences till now for those services used in our own workflow. We would certainly taking this thinking into account in our new grand plan for technologies supporting our people.
Google’s BeyondCorp is probably the most known implementation of Zero trust Security and they certainly achieved some degree of success. Although our organisation is nowhere as resourceful as Google, I am still optimistic about our potential future implementation of Zero Trust Security: There are not that many services we have left that does not fit in our new grand plan. Changing the few left should be much easier than having to deal with hundreds of them in a larger organisation, right? Tell us what you think and, even better if you can comment on your own experience implementing Zero Trust Security.
