BCP during COVID
What BCP means now?
Everyone thought at business continuity in terms of IT service provider's locations getting failed. Hence BCP was planned and tested at various levels - intra city, inter city, inter country, etc. Covid however has proved that BCP should be something more than this. We have now faced an unpredicted scenario – “All locations are Down - Business operations to be done from everyone's home”. Ultimately, BCP got redefined after Covid. This Case study is based on how BCP was implemented for a global provider of financial market data and infrastructure
IT Services to Client
The IT service provider provided testing services for various product areas (Trading, Wealth Management, Risk management, Investment banking, etc.).
Team Size: 120+
Contract type: T&M
Challenges Faced
Maintaining the service levels across applications
Product users escalating availability issues
>75% of the team were desktop users. Work from Home(WFH) enablement was a top priority
Potential Security issues in home network
Maze Ransomware attack in the IT service provider Organization
Internet availability and speed at home- broadband/4G/mobile data
System slowness for WFH enabled employees due to additional antivirus agents running in background
Virtual Desktop Infrastructure (VDI) slowness
Attrition
The Reaction
Only critical applications were serviced during early Covid times. Service levels were maintained within target
Certain critical applications experienced intermittent availability issues and were escalated by product users. Root cause was Memory leak issues which went undetected during short test runs. Improved Performance tests and test coverage
Desktops and Laptops delivered to employees at their home. This was not an easy task. Mobilised from the existing assets, additional procurement done and managed to transport them to employee’s home.
Implemented Bring Your Own Device (BYOD) policy and enabled employees to use their own laptops/desktops that were meeting the configuration needs after getting client approval
WFH enablement progress, employee absence, productive time, etc. reviewed by Senior Management daily
For WFH not enabled employees, separate project "COVID" created for logging time sheets
Ransomware issue - mail attachment restriction between client ID and Service provider ID, temporary links to key internal systems - timesheets, payroll, etc.
Provisioning of Data cards, reimbursement for employees buying new internet connections
VDI - additional RAMs in the VMs for critical employees
Mandatory antivirus and security patched in all laptops/desktops
Additional Helpdesk numbers/POCs for software/hardware issues in laptop/desktop
Timely off boarding of employees - deferred collection of assets based on online declaration
Best practices implemented
Keep the team posted on Covid prevention measures - Dos & Don’ts, BCP status, Lockdown updates, Location readiness, etc.Not compromising on activities like Project Management Reviews, Internal audits, etc.
Communications with clients were transparent from the start during Ransomware attack/COVID
Team's productive time shared to client stakeholders’ daily
When things were back to normal, delivery process was enhanced - Customized Engineering methodology with improved Test standards, additional Quality Engineer roles in scrum team, increasing Automation test coverage target, etc
What was the Outcome?
100% employees enabled for WFH in 1.5 months
Un-interrupted essential services to clients initially followed by making up to Business As Usual
Enhanced client relationship
Improved Wallet shares
Recommendations
Avoid vulnerabilities with VPN by using VDI
Transparency with client on the actions being taken and providing daily updates during difficult times
Bill to client only for the productive time spent on deliverables
Maintain 100% security compliance in laptops/desktops for Data Loss Prevention(DLP)
100% Cloud enablement across client's portfolio
Further harden network security