Barathwaj Maadhavan
AI Engineer · Forward Deployed Engineer
AWS Cost Allocation & Reporting Automation
Sole developer – automated shared cloud cost allocation for a multi-client AWS portfolio.
Overview
The AWS Cost Allocation & Reporting project was undertaken to improve the accuracy, transparency, and efficiency of monthly cloud cost allocation for a portfolio of approximately five to six clients sharing the same AWS resources and infrastructure.
Initially, the organisation relied on high-level AWS usage-percentage reports and a combination of Power BI dashboards and complex Excel spreadsheets spanning seven to eight sheets, containing numerous formulas and data sources to calculate and consolidate monthly costs for each client. This process was managed by the Production Support Engineering team and involved significant manual effort. More importantly, the existing approach used rough usage-based estimates that did not accurately reflect individual client consumption — some clients were charged approximately $100–$200 even during months when they had not processed any data, leading to dissatisfaction and highlighting the need for a more precise cost allocation mechanism.
The Challenge
The existing cost allocation process suffered from two distinct problems:
- Operational inefficiency — a highly manual monthly workflow involving AWS reports, Power BI dashboards, and multi-sheet Excel calculations.
- Inaccurate allocation — rough usage-based estimates did not reflect actual client consumption, resulting in unfair charges and client dissatisfaction.
The project was therefore split into two phases — first automating what existed, then fixing what was fundamentally wrong with the methodology.
What I Built
Phase 1 — Automating the Existing Workflow
I developed a fully automated Python script that ran in the middle of each month. It:
- Retrieved raw cost and usage reports from AWS.
- Pulled relevant usage data from internal processes.
- Applied the existing cost-splitting formulas.
- Consolidated the results into monthly reports.
- Automatically distributed the reports to the account head via email.
This eliminated the need for Production Support Engineers to manually extract data, maintain complex Excel calculations, and use Power BI to roll up monthly costs — improving operational efficiency and removing repetitive manual work.
Phase 2 — Tag-Driven Accurate Allocation
While Phase 1 streamlined the process, it did not fix the underlying allocation methodology. In Phase 2, the project evolved into a more accurate, tag-driven cost allocation model that recognised different clients and infrastructure types required different cost-splitting approaches.
I introduced a tagging mechanism for AWS servers and resources, where each tag determined how costs should be allocated:
- Equal allocation — costs split evenly among all clients.
- Hybrid allocation — a combination of equal and usage-based splits (e.g. 50% equally, 50% by actual usage).
- Direct assignment — costs assigned to a specific client.
To support accurate usage attribution, background processes running on the shared servers were used to audit and identify processing activity associated with different clients. This established a clearer relationship between resource utilisation and individual client consumption.
Results & Impact
- Eliminated manual effort — replaced multi-sheet Excel workflows and Power BI roll-ups with a single automated script.
- Improved allocation accuracy — tag-driven model reflected actual client consumption.
- Fairer billing — clients that processed no data were no longer charged for idle infrastructure.
- Positive client feedback — revised charges were significantly lower in certain cases and better aligned with actual usage.
- Finance-ready reporting — detailed monthly cost reports delivered directly to Finance for client billing.
Key Learnings
This project taught me that automation alone is not enough — automating a flawed process simply produces flawed results faster. The real value came from questioning the underlying methodology and redesigning it around accurate, verifiable data. I also learned the importance of working closely with Finance and Support teams to understand both the technical and business sides of cost allocation.