What a Software Architecture Review Actually Covers, and When It Is Worth Running One

Every serious review framework converges on the same handful of areas, which is a useful sign that the model is real rather than a vendor invention. Amazon's Well-Architected Framework uses six pillars: operational excellence, security, reliability, performance efficiency, cost optimisation and sustainability. Microsoft's Azure Well-Architected Framework uses five, dropping only sustainability. Two competing cloud vendors independently landing on the same five pillars is the closest thing this field has to a standard. What matters more than the pillar names is the question each area is trying to answer about your business. A review that returns generic best practice has failed. A review that tells you which specific thing will break first, and what it will cost you when it does, has not. | Review area | The question it answers | |---|---| | Business drivers and quality priorities | What does this system have to be good at, and what is it allowed to be mediocre at? | | Reliability | When this fails, what fails with it, how fast do you notice, and how do you get back? | | Security | Who can reach your data, what happens if a credential leaks, and what is exposed to the public internet right now? | | Operational readiness | Can somebody other than the original developer diagnose a production problem at 2am? | | Performance and scalability | Does this hold up at the growth you actually expect, not a hypothetical one? | | Cost | Which architectural decisions are generating your bill, as distinct from which settings are misconfigured? | | Change cost and coupling | How much does the next feature cost, and why is that number rising? | | Sensitivity and tradeoff points | Where does a small change break something big, and where do two goals fight each other? | | Data and integration boundaries | What happens when a third-party service you depend on changes or disappears? | | Remediation plan | What do you fix first, and what can safely wait? | The last two rows are where reviews earn their money. Most systems fail at a boundary somebody assumed was stable.

The AWS Well-Architected Review runs in five stages, and it is the most widely used commercial format because it is the most operationally simple. First, assemble the group: architects, developers, someone from operations, and importantly a person from the business who can say what the system is for. Second, run a workshop working through a structured question set for a single defined workload, not the whole estate. Third, classify the findings as High Risk Issues or Medium Risk Issues and rank them. Fourth, build an action plan against the ranked risks. Fifth, repeat, which AWS suggests doing every three to six months. The Azure equivalent is more prescriptive about volume. Microsoft's Well-Architected Review is a structured assessment of roughly 60 questions drawn from the key recommendations of each pillar, producing tailored recommendations with links to supporting guidance that can be exported into your own planning tools. Scope discipline is what separates a useful review from an expensive one. Practitioner accounts put a self-service review of a single workload against all six AWS pillars at around two to four hours for an experienced architect, though that figure comes from practitioner blogs rather than AWS documentation. The point is not the exact number. The point is that reviewing one workload well is a day's work, and reviewing everything at once is a project that never finishes. Google's Site Reliability Engineering book documents a different trigger. Its Production Readiness Review runs at a handover boundary, before the SRE team takes on production responsibility for a service, with phases covering engagement, analysis, refactoring, training, onboarding and continuous improvement. Google is candid that its simple review model did not scale and had to evolve into frameworks and a platform, which is a useful warning to anyone hoping a review is a one-off event.

The Architecture Tradeoff Analysis Method, developed at Carnegie Mellon's Software Engineering Institute, is the rigorous vendor-neutral option, and it produces something a checklist review cannot. ATAM runs nine steps across two phases, with phase one typically taking a day followed by a two to three week gap before phase two. Phase one covers presenting the method, presenting the business drivers, presenting the architecture, identifying the architectural approaches in use, building a quality attribute utility tree, and analysing the architecture against the highest priority scenarios. Phase two widens the group, brainstorms and reprioritises scenarios, reanalyses, and presents the results. The utility tree is the step commercial reviews usually drop, and it is the reason ATAM is worth knowing about. It forces the business goals to be decomposed into concrete, ranked scenarios before anybody looks at a diagram. Instead of "the system should be fast", you get "a warehouse worker scanning a pallet at peak intake must see a confirmation within one second, and this matters more than the reporting dashboard loading quickly". Every technical finding is then scored against that, which is what stops a review from becoming a list of the reviewer's preferences. ATAM's four outputs are risks, non-risks, sensitivity points and tradeoff points. A sensitivity point is a piece of the system where a small change materially moves a quality attribute. A tradeoff point is one that is sensitive to two attributes pulling in opposite directions, where improving one degrades the other. Those two categories are the genuinely valuable artefacts, because they are the map of where your system is fragile in ways nobody has written down.

You will hear that a defect costs 100 times more to fix in production than in requirements, usually credited to the IBM Systems Sciences Institute. That claim does not survive checking, and any consultant who repeats it without qualification is telling you something about their sourcing. The figure traces to Roger Pressman's 1987 textbook *Software Engineering: A Practitioner's Approach*, which attributed the numbers to course notes from the IBM Systems Sciences Institute. Laurent Bossavit investigated and found the institute was an internal employee training programme rather than a research body, and that no supporting data was ever published. Hillel Wayne reached the same conclusion independently. The Register covered the finding on 22 July 2021 under the headline that everyone cites the research but the study might not even exist. There is real data underneath the myth, and it makes a better argument. Barry Boehm published cost-to-change data in *IEEE Transactions on Computers* in 1976 drawing on project data from TRW, corroborated by figures from IBM, GTE and Bell Labs, and expanded it in *Software Engineering Economics* in 1981. His finding was roughly 100 to 1 cost growth, but with two qualifiers that get dropped in the retelling: on large projects, and on badly run ones. Boehm later moderated it further, noting the curve is much flatter on projects of 50 people or fewer, and wrote in 2004 that no empirical data was found for small agile projects. Read properly, Boehm supports architecture review more strongly than the myth ever did. The steepness of the cost curve is a property of how a project is run, not a law of physics. That is precisely the variable a review changes.

The best-documented figure on software quality economics comes from NIST Planning Report 02-3, prepared by the Research Triangle Institute in May 2002, which estimated that software defects cost the US economy USD 59.5 billion a year, around 0.6 per cent of GDP at the time, which is roughly A$90.8 billion at July 2026 rates. It judged USD 22.2 billion of that, about A$33.9 billion, feasibly recoverable through better testing and verification infrastructure. The report is 24 years old, which is worth saying, but it remains a government-commissioned study with a published methodology rather than a vendor estimate. More recent numbers describe the same problem from the inside. The Consortium for Information and Software Quality put the cost of poor software quality in the US at USD 2.41 trillion in its 2022 report, about A$3.68 trillion, with roughly USD 1.52 trillion of that, around A$2.32 trillion, being accumulated technical debt. The figures that matter most to a business owner are the ones about where developer time goes. Stripe's 2018 *Developer Coefficient* study, run with Harris Poll across executives and developers in more than 30 industries, found developers spending 13.5 of 41.1 hours a week on technical debt, roughly a third of the week, with 3.8 hours specifically on fixing bad code. McKinsey's 2020 survey of 50 CIOs at financial services and technology firms with revenue above USD 1 billion, about A$1.53 billion, found 10 to 20 per cent of the technology budget for new products being diverted into resolving technical debt, and 60 per cent of those CIOs saying the debt had risen perceptibly over the previous three years. That sample is small and narrow, so treat it as a signal rather than a census. Stack Overflow's 2024 developer survey found 62 per cent of developers naming technical debt as a problem, twice the share of the next two most frustrating issues. The direction of every one of these measures is the same, even where the magnitudes disagree.

If your system was built quickly with AI coding tools, a review is looking for a specific and well-documented set of failures rather than general untidiness. Veracode's 2025 GenAI Code Security Report tested 80 curated coding tasks across more than 100 large language models and found that when a model had a choice between a secure and an insecure implementation, it chose the insecure one 45 per cent of the time. Java was the worst language in the study at a 72 per cent security failure rate. That is the controlled-study result. The field result matches it. Escape scanned 5,600 publicly available vibe-coded applications on 29 October 2025 and found 2,038 highly critical vulnerabilities, more than 400 exposed secrets and 175 distinct instances of exposed personally identifiable information, including bank account data. More than 50 database tables accepted unauthenticated writes, among them tables named payments, orders and transactions. Escape used deliberately passive scanning to stay within ethical bounds and states plainly that its findings reflect only a subset of exploitable issues, so those numbers are a floor. The same class of problem produced CVE-2025-48757, where missing or insufficient database row-level security policies exposed 303 vulnerable endpoints across 170 websites built on one AI app platform, found by examining homepages alone. The vendor disputes the record on shared-responsibility grounds, and the researcher who disclosed it worked for a competitor at the time, both of which are worth knowing. The underlying misconfiguration pattern is not in dispute. The practical consequence is that a review of an AI-built system starts with a data-exposure sweep before it looks at anything else: which tables are readable without authentication, which functions are callable with a public key, which API keys are sitting in the client-side bundle. Those are cheap to check and catastrophic to miss, and they are invisible from the outside because the application looks finished.

Run a review at a boundary where the cost of being wrong steps up, not on a calendar reminder. The four frameworks disagree about timing in an instructive way, and their disagreement maps neatly onto real business triggers. AWS recommends reviewing every three to six months as standard operations, which suits a system under active development on cloud infrastructure. Google's SRE model triggers a review at handover, before a team takes on production responsibility for a service somebody else built. ATAM triggers when quality requirements are contested or when architectural tradeoffs have not been explicitly agreed by the people who will live with them. ThoughtWorks, through Neal Ford and Rebecca Parsons, argues for continuous automated checks instead: architectural fitness functions that give an objective assessment of a chosen characteristic and run in the deployment pipeline, turning governance into a test suite rather than a meeting. Translated into business terms, the moments worth paying for a review are these. You have inherited a system, whether by acquisition, by a developer leaving, or by taking over an internal tool nobody owns. You are about to spend significantly more on the system than it cost to build. You are moving something from prototype to real customers and real money. Your change costs are rising and nobody can explain why. Or a specific decision is being argued about internally and the argument is not converging. The case for reviewing an inherited system before you commit money to it is the strongest of these, because the alternative is discovering the constraints while you are already paying for the build. If the outcome you want is a decision about whether to keep or replace the system, the rebuild or refactor question is a different exercise with a different rubric. A review tells you what is there and what it will cost you. It is the input to that decision, not the decision itself. The case against is equally simple and worth stating. If the system is small, well understood by people still working on it, not holding sensitive data, and not about to grow, a formal review is probably not where your money goes furthest.

Is an architecture review the same thing as a code review? No. A code review looks at individual changes for correctness, readability and style, usually as part of everyday development. An architecture review looks at the decisions that are hard to reverse: data model, service boundaries, authentication, hosting, integration points. You can have clean code inside a badly structured system, and that combination is common in fast AI-assisted builds. How long does a review take? It depends entirely on scope. Practitioner accounts put a single defined workload against all six AWS Well-Architected pillars at roughly two to four hours for an experienced architect, though that figure is blog-sourced rather than official. A full ATAM evaluation is heavier: nine steps across two phases, with phase one taking about a day and phase two following two to three weeks later. What do I get at the end of it? A prioritised list of risks classified by severity, each tied to a business consequence, plus a remediation plan saying what to fix first. Under the AWS format the risks are labelled High Risk Issues and Medium Risk Issues. Under ATAM you also get sensitivity points and tradeoff points, which map where a small change breaks something and where two goals conflict. Does a review find security vulnerabilities? It finds architectural ones, which are usually the serious ones. Missing row-level security, tables readable without authentication, secrets in client-side bundles, functions callable with a public key. Escape's October 2025 scan of 5,600 vibe-coded applications found 2,038 highly critical vulnerabilities and more than 400 exposed secrets, which are exactly this class of problem. A review is not a substitute for penetration testing, which probes a running system rather than its design. Is it true that bugs cost 100 times more to fix in production? That specific figure is not supported. It traces to a 1987 textbook citing unpublished course notes from an IBM internal training programme, and no data was ever published. Barry Boehm's real 1976 data found roughly 100 to 1 growth on large, badly run projects, and he later noted the curve is much flatter on projects of 50 people or fewer. My app was built with an AI platform. Is a review worth it? Usually yes, and the data-exposure checks are the part that pays for itself. Veracode's 2025 report found models choosing an insecure implementation 45 per cent of the time across 80 tasks and more than 100 models. An AI-built application typically looks finished from the outside while the access controls behind it were never configured, so external appearance tells you nothing.