Imagine logging into AWS Control Tower one day and being greeted with the screen below. A full page drift error is a rare scare when trying to maintain a well-governed multi-account AWS environment. When your landing zone deviates from its intended state, it can create security gaps, compliance issues, and operational headaches. Let’s explore what drift means in AWS Control Tower, why it occurs, and how to effectively manage it.
What is Control Tower Drift?
Drift occurs when your AWS Control Tower environment’s actual state differs from its expected configuration. This can happen at various levels - from individual resources to entire organizational units (OUs). Think of it like a carefully arranged bookshelf slowly becoming disorganized as people borrow and return books without following the original system.
In technical terms, drift manifests when:
- Service control policies (SCPs) managed by Control Tower are changed directly in AWS Organizations,
- Control configurations (like AWS Config rules or Security Hub controls) are modified outside of Control Tower,
- Resources or settings are created in an account in ways that bypass Control Tower’s governance,
- Accounts are moved to different OUs or removed from the organization without using Control Tower,
- Control Tower baseline resources (such as the Security OU or audit roles) are deleted or altered manually.
Some drift scenarios are minor, but others can be critical. For instance, deleting the default Security OU (which holds core logging/audit accounts) will effectively break Control Tower functionality until it’s resolved. In short, a drifted environment is no longer fully aligned with the safe, compliant baseline that Control Tower established.
Common Causes of Drift
Understanding why drift occurs is crucial for preventing it. The most frequent causes include:
- Manual Changes: Direct changes to accounts, OUs, or policies made outside Control Tower bypass Control Tower’s controls and introduce drift.
- Automated Tools: Scripts, CI/CD pipelines, or Infrastructure-as-Code tools that aren’t integrated with Control Tower may create or modify accounts and resources in ways that ignore Control Tower’s governance and cause drift.
- Inconsistent Updates: Not all parts of an environment get updated at the same time. If you delay applying Control Tower updates or enabling new controls, some accounts may remain on older settings while others use the updated rules. This lack of uniformity can lead to drift over time if not reconciled.
Detecting Control Tower Drift
AWS Control Tower automatically monitors your environment for signs of drift. It focuses on changes in your AWS Organization and control policies – for instance, accounts moving between OUs or modifications to Control Tower-managed SCPs. Such events trigger alerts in the Control Tower console and send Amazon SNS notifications to the audit account.
However, Control Tower’s drift detection doesn’t cover everything. It won’t catch changes inside individual accounts that fall outside its controls. To cover these gaps, many organizations set up additional custom monitoring. For example, AWS Config rules or EventBridge events can be used to detect configuration changes that Control Tower might miss. This way, if a critical setting is altered outside of Control Tower’s view, your custom checks will still catch it.
Importantly, while your landing zone is in a drift state, certain Control Tower features (like enrolling new accounts) are unavailable. That’s another reason to resolve drift quickly.
Best Practices for Drift Management
To effectively manage Control Tower drift, consider implementing these strategies:
Preventive Steps
- Strongly avoid making changes to Control Tower related components outside of Control Tower at all costs, unless you know exactly what you are doing or have guidance.
- Enforce strict change management for any org-wide or account-level changes. Require that new accounts, OU changes, and control modifications go through automated pipelines or Control Tower’s mechanisms (rather than ad-hoc manual steps).
- Use AWS Organizations SCPs to prevent unauthorized alterations.
- Maintain clear documentation of your baseline and regularly train teams on Control Tower governance, so everyone knows how to make changes the right way.
Access Control
- Almost all routes that lead to Control Tower drift is via the management account where not only Control Tower lives but also AWS Organizations and other services that manage and vendor out accounts.
- Ensuring the principle of least privilege is followed using IAM. Both in the number of staff members who can use the management account and also what changes they are allowed to make reduces the possibility of making unintentional changes that could trigger drift.
Remediation Strategies
- React swiftly. If drift occurs ensure treat it as an incident to ensure that your landing zone spends the shortest time possible in a drift state.
- Prepare clear runbooks for common drift scenarios.
- Use Control Tower’s own tools (like Re-register OU or Landing Zone Reset button (see above)) to restore your environment to its expected state whenever possible.
- It is also possible for only a single account in a “tainted” state, which often occurs when default VPCs are manually added after initial setup, preventing account updates. To fix this, delete the problematic default VPC through the VPC console, then update the account in Control Tower to allow proper reconfiguration of network resources.
- Regularly test your remediation process (and ensure you have rollback options) so you can fix drift quickly when it happens.
Continuous Improvement
- Periodically review drift incidents to spot patterns, address root causes, and run regular audits for compliance (e.g. add or adjust controls to prevent recurrences).
- Keep your environment up-to-date and adapt your controls as AWS releases new features or updates.
Our door is always open
A successfully automated implementation of Control Tower often requires specialised expertise and experience. Consider working with an AWS partner, such as The Scale Factory, who have deep knowledge and practical experience in deploying and optimising AWS Landing Zones for SaaS organisations. We can provide valuable guidance, best practices, and bring the experience gained by delivering 100+ Landing Zone implementations, including customised solutions tailored to SaaS requirements.
We also have another post about landing zones, diving into more detail why you should use one.
Conclusion
AWS Control Tower drift is inevitable in large cloud environments, but it doesn’t have to derail your operations. By understanding its causes, detecting issues early, and responding with a solid remediation plan, you can maintain a secure and compliant environment even as things change. Treat drift management as a continuous effort that evolves with your AWS usage. With the right approach, whenever Control Tower drifts, you’ll be ready to correct course quickly.
Like the sound of all that but want some expert help implementing Control Tower in your AWS estate? Our Foundational Landing Zone package includes a bespoke Control Tower design and installation by our experienced AWS consultants for a simple fixed price.
This blog is written exclusively by The Scale Factory team. We do not accept external contributions.
