Request for Proposal (RFP): Redemption and Reallocation of Nomad Collateral and Protocol Reserves for FRAX Market Enhancement

Solidity Labs Proposal: Redemption and Reallocation of Nomad Collateral and Protocol Reserves for FRAX Market Enhancement

Overview

Solidity Labs would like to help restore liquidity to the Moonwell protocol on Moonbeam. We will do this by leveraging our deep knowledge of smart contracts, security tools, and our network of trusted security professionals.

Before diving into the proposal steps and approaches, it is important to describe the current state of the protocol, and what must remain true as these corrections occur. When a user’s collateral factor dips below 100%, as defined in the Comptroller, other users can liquidate them, closing up to the specified percentage of their position defined by the close factor, and repaying their debt. When a liquidation occurs, the liquidated user has their debt paid off, and the liquidator receives a portion of the collateral above the debt value as a bonus for taking this action. When Nomad was exploited, this normal liquidation process was circumvented as the collateral immediately lost 80% of its value, and the Chainlink oracles were oblivious to this change, preventing liquidations of positions that now had bad debt. In order to correct this, and put the protocol in the most correct state, the users with bad debt must be force liquidated by governance, have their collateral seized, and then liquidate said collateral with a multisig of trusted community members.

During this liquidation, there are protocol invariants that must remain intact. Total borrows must equal the sum of all user borrows, total supply of mTokens must equal the sum of all user mToken balances, and the exchange rates for each affected mToken must remain constant. Keeping these invariants intact is crucial to the success of the proposal and the health of the protocol.

Trusted Signer

In order to facilitate the seamless execution of the liquidation of Nomad assets, a member of the Solidity Labs team will be on the multisig wallet for the recovery effort. The team has been on many multisig wallets before and executed parameter changes, swaps, governance actions, and recovery efforts during war rooms. A fresh wallet will be created with high-security standards to avoid compromise.

Existing Bad Debt and Collateral

There are currently 81 accounts with bad debt. The following table describes the borrowed assets that will likely never be repaid, and the corresponding non-nomad supplied assets that can be seized.

Token Seizable Supplied Assets (non .mad) Borrow
FRAX 0.0028 3,015,166.39
xcDOT 3.71 493,252.97
GLMR 30,318.68 5,333,685.56

Solidity Labs will work closely with Gauntlet and Moonwell delegates like @Curly to adjust Interest Rate Models and incentives. This is needed to ensure supply rates are unchanged as both supply and borrow rates will decrease from the bad debt write-off in impacted markets. The FRAX market will see negligible rate changes as the recent Gauntlet proposal moved the interest rate model to a maximum of 1%. However, xcDOT and GLMR will see decreases in rates for both suppliers and borrowers as the total borrows decrease, lowering utilization.

Paths Forward

This post describes at a high level the actions that the Moonwell community will need to take to recover the Nomad assets and restore liquidity. Two paths forward will be proposed, each with its own set of tradeoffs around speed, and thoroughness. The more thorough proposal will take longer, involve more risk, and solve the underlying issues as best as possible. While the less thorough proposal will take less time, involve less risk, and not fully address the underlying issues.

Standard Approach

In order to stop bad debt from further accruing interest, the accounts with bad debt must be manually liquidated. The mToken implementations must be updated to allow a privileged actor the ability to write off liabilities of accounts with bad debt to zero. These accounts will then have their collateral transferred to a trusted address. All of these actions must take place while still preserving the core protocol invariants. However, once the bad debt is subtracted from the user balances and the total borrow amount, the exchange rate of the mToken will change. This means new mToken contracts will need to be created which preserve the rates across time and continue accruing even after the bad debt has been written off.

Once these accounts have their assets and liabilities written off, it will cause a downward change in both supply and borrow rates for the GLMR and xcDOT markets as the losses are realized. To manage this, Gauntlet will need to make a recommendation of the new Interest Rate Model for markets affected by these write-downs.

Comptroller Fixer Contract

This fixer contract will allow the timelock to pass an array of users. This array of users will then be iterated over, and for each user, each mToken they possess will be iterated over, and if they have a supply or borrow balance, call into the mToken contract and zero that users’ associated borrow balance, and send all corresponding supply mToken balance to the community multisig.

mToken Fixer Contract

The Comptroller can call the fixUser function on the mToken Fixer, which will zero a user’s borrow balance, subtract the previous balance from the totalBorrowBalance, and send the users’ mToken balances, to the community multisig. Each amount of borrowed balance written off will be added to another state variable that will be added to the current cash of the contract. This will ensure the key protocol invariant remains intact and the exchange rate of the markets with bad debt does not reprice downwards as this debt is written off and the corresponding amount of cash is not returned to the market.

Formal Verification

In order to verify that the upgrade and liquidate proposal does not violate the core invariants of the Moonwell Protocol, Solidity Labs can write formal specifications and rules for the updated Comptroller and mToken Fixer using the Certora Prover to demonstrate the new code does not violate any protocol invariants.

Storage Slot Clashes

In order to verify the proposed changes do not write to storage incorrectly, storage checking tools will be used to ensure that the offsets of the mToken Fixer contract are not different from the existing code. This layer of testing will give us and the community confidence in the correctness of the proposal and code before going to the audit stage.

Integration Testing

These changes to the protocol are nontrivial and will require significant modifications. In order to ensure these changes do not cause further issues, significant integration testing will be done to verify the correctness of the codebase after the upgrade and governance proposal. Testing types and assertions include but are not limited to:

  • Application of Forge Proposal Simulator tool which has been developed to ensure the safety of all types of governance proposal changes.
  • Check that no account health factors change after each proposal, except for the accounts that are being liquidated.
  • Assert liquidated accounts have no mToken balance.
  • Validating no configuration changes inside the Comptroller occurred.
  • Ensuring no balance change occurred, except in the specified accounts to be liquidated.
  • Verifying that the multisig received both the Nomad collateral and all other mToken collateral from accounts with bad debt and reserves.

Multisig Simulation

All multisig actions that require interacting with Moonwell contracts will have calldata programmatically generated using the Forge Proposal Simulator. Additionally, integration tests will be created to ensure that each action the multisig takes that interacts with the protocol is correct.

Peer Reviews and Auditing

Before proceeding, our trusted network of security researchers, engineers, and advisors will review our specifications and code to ensure the intended outcome of the proposal for all parties and actors in the system. Due to the technical complexity of this approach, we will engage in two audits, the first with Halborn and the second with Code4rena. Code4rena will perform a governance proposal audit to ensure the safety of the proposal.

Proposals

Because of the complexity of these operations, two governance proposals will need to be passed to complete the write-down of these accounts. After these proposals are executed, the trusted community multisig will need to be used to perform actions relating to exchanging certain cryptocurrencies for others and bridging the Nomad assets back to mainnet. Solidity Labs will advise and review these proposed actions, ensuring funds are not lost during this crucial time.

Proposal 1

Currently, the maximum number of actions a proposal can execute is 25. However, this will need to be increased in order to handle the sheer volume of actions that will be required in order to fix the bad debt accounts in proposal 2. This proposal will increase the maximum number of proposal actions to 1,000 so that future proposals are not bottlenecked on the number of actions they can take. During this proposal, all Nomad reserves will be transferred to the community multisig. This will allow the KYC process to begin before the second proposal has taken place.

Proposal 2

This proposal will remove all collateral mTokens from users with bad debt from all markets, sending them to the community multisig. In order to allow mToken collateral sweeping and bad debt write offs, the mTokens will have their implementations upgraded. This will stop further issues and to allow the exchange rates of the assets to stay the same post proposal. Interest Rate Models may need to be changed in order to stop rate spikes due to changes in supply and borrow amounts.

Post Proposal 2

The community multisig will redeem the mTokens for their corresponding amounts of Nomad and underlying assets. If there are any non-nomad underlying assets such as GLMR, will be added to the reserves. Any Nomad assets will go through the recovery process and be swapped to FRAX on mainnet and bridged back to moonbeam where they will be added as reserves to enhance market liquidity.

Alternative Approach

Instead of fixing the bad debt with a contract that will manipulate storage, and stop it from accruing further interest, there exists a simpler path. This solution involves only pulling the Nomad collateral from the mToken, and letting the bad debt in both xcDOT and GLMR markets continue accruing. With this solution, most of the FRAX bad debt can be repaid on behalf of the borrowers with bad debt. Taking this course of action will allow further accrual of ~49,000 xcDOT and 320,000 GLMR in interest per year at current rates from existing bad debt.

This approach has much less technical risk and involves less custom code. It may be possible to handle all of the governance steps for this proposal in a single governance proposal. However, with such a low amount of actions per proposal in the current governor, and the code not yet written, it may require a second proposal to complete. With this approach, liquidity will be restored more quickly to the FRAX market, while still allowing further bad debt to accrue in both the GLRM and xcDOT markets.

Comptroller Fixer Contract

The alternative approach will not require the Comptroller Fixer contract to function as individual and global values for each mToken will not be updated.

mToken Fixer Contract

An upgraded mToken implementation will be needed, but only for the duration of the proposal. Before proposal completion, the mToken logic contract for each respective .mad asset will be swapped back to its original implementation.

Formal Verification

Formal verification is not needed for this approach due to the ephemeral nature of the upgrade.

Storage Slot Clashes

Storage slot clashes will be checked in this proposal. However, due to the ephemeral nature of the upgrade, and not writing to storage, this is not a security concern.

Integration Testing

In order to ensure these changes do not cause further issues, Solidity Labs will engage in significant integration testing to verify the correctness of the codebase after the governance proposal.

  • Application of Forge Proposal Simulator tool which has been developed to ensure the safety of all types of governance proposal changes.
  • Check that no account health factors change after each proposal.
  • Assert that no configuration changes inside the Comptroller occurred.
  • Validate no balance change occurred, except in .mad mToken underlying balances.
  • Verify the multisig received the Nomad collateral and reserves.

Multisig Simulation

All multisig actions that require interacting with Moonwell contracts will have calldata programmatically generated using the Forge Proposal Simulator. Additionally, integration tests will be created to ensure that each action the multisig takes that interacts with the protocol is correct.

Peer Reviews and Auditing

Before proceeding, our trusted network of security researchers, engineers and advisors will review our specifications and code to ensure the intended outcome of the proposal for all parties and actors in the system. Halborn will be engaged to audit this proposal.

Proposal 1

Currently, the maximum number of actions a proposal can execute is 25. However, this will need to be increased in order to handle the volume of actions that will be required. Actions include upgrading the mTokens, sweeping the underlying, and returning the implementations back to their original contracts. This proposal will increase the maximum number of proposal actions to 1,000 so that future proposals are not bottlenecked on the number of actions they can take. During this proposal, all Nomad reserves will be transferred to the community multisig so that the KYC process can begin before the second proposal has taken place.

Proposal 2

This proposal will remove all nomad assets from nomad mTokens and send them to the community multisig. After all mToken collateral has been swept to the community multisig, the mTokens will have their implementations changed back to the old mToken implementations to avoid further issues. Interest Rate Models may need to be changed after this proposal to stop rate spikes after borrows are repaid.

Post Proposal 2

The community multisig will redeem the mTokens for their corresponding amounts of Nomad and underlying assets. If there are any non-nomad underlying assets such as GLMR, they will be added to the reserves. Any Nomad assets will go through the recovery process and be swapped to FRAX on mainnet and bridged back to moonbeam where they will be added as reserves to enhance market liquidity.

Timeline

Executing the standard approach is estimated to take seven weeks from proposal start to finish. Executing the alternative approach is estimated to take five weeks. In both proposals, Solidity Labs will immediately submit KYC documents to the coinlist Nomad recovery site as this process is estimated to take between one to four weeks.

Standard Approach

Action Estimated Time
Nomad KYC 1 - 4 weeks, non blocking
Proposal One 1 week
mToken and Comptroller Upgrade 2 weeks, non blocking
mToken and Comptroller Upgrade Review 2 weeks, blocked on upgrade
Proposal Two 1 week, blocked on mToken
Nomad asset recovery and swaps 1 week, blocked on proposal two and KYC
Total Time 7 weeks
  • Proposal one will increase the proposal count and pull the nomad reserves. This will take one week.

  • Writing and modifying the mToken and Comptroller to pay off the bad debt. This will take at least two weeks. Once completed, it will require at least two weeks of review from auditors and external security researchers.

  • Proposal two will be ready to submit on chain at week five, there will be a one week window for voting, completing in the sixth week. The KYC should have been completed, and the assets will be able to be unbridged from Moonbeam back to ethereum mainnet where the Nomad assets can be claimed. A separate multisig wallet will be created on mainnet with the same signer set so that the Nomad assets will be in the multisig at all times.

  • Underlying assets will be claimed on Ethereum mainnet and swapped for FRAX. This FRAX will be bridged back to Moonbeam using the FRAX Ferry. This phase is expected to take another full week.

  • All of the assets will be bridged back to Moonbeam and added as reserves to the market to boost liquidity.

  • Total Proposed Time: seven weeks

Alternative Approach

Action Estimated Time
Nomad KYC 1 - 4 weeks, non-blocking
Proposal One 1 week
mToken Creation 1 week, non-blocking
mToken review 1 week, blocked on mToken creation
Proposal Two 1 week, blocked on mToken review
Nomad asset recovery and swaps 1 week, blocked on proposal two and KYC
Total 5 weeks
  • Proposal one will likely need to increase the proposal count and will have to pull the nomad reserves. This will take one week.

  • Creating the mToken that will allow the transfer of all Nomad collateral to the multisig is expected to take one week to write, and once completed, will require one week of review from auditors and external security researchers.

  • Proposal two will be ready to be submitted on chain at week three, there will be one week for voting, completing in the fourth week. The KYC should have been completed, and the assets will be able to be unbridged from Moonbeam back to ethereum mainnet where the Nomad assets can be claimed. A separate multisig wallet will be created on mainnet with the same signer set so that the Nomad assets will be in the multisig at all times.

  • Underlying assets will be claimed on Ethereum mainnet and swapped for FRAX. This FRAX will be bridged back to Moonbeam using the FRAX Ferry. This phase is expected to take another full week.

  • All of the assets will be back on Moonbeam and can be added as reserves to the market to boost liquidity.

  • Total Proposal Time: five weeks.

The total amount of time required to complete the standard approach is seven weeks. If the alternative approach is taken, it will take two weeks less because less code will be written and fewer reviews are needed to ensure the safety of the proposal.

Tradeoffs and Risks

There exists no perfect solution to the problem as defined by the RFP. Both proposed solutions exist in a tradeoff space of speed and correctness. The standard approach is the most correct, but it comes with additional risk. The alternative approach is the least correct but restores liquidity faster. Our goal in writing this document is to enumerate the paths forward and allow the community to decide which path is best for them.

The standard approach outlined throughout this document carries increased smart contract risk around the upgrade, which could cause further issues. However, this risk will be mitigated by extensive integration testing, internal security reviews, formal verification, auditing, and review by other experts of Compound V2-based lending protocols. This standard approach stops additional interest from accruing and fixes these markets, at least from an interest accrual perspective.

If the alternative approach is taken, there is much less smart contract and execution risk. The mToken contracts will remain unchanged after the collateral is withdrawn from the markets. No patching of mTokens will need to occur to preserve key protocol invariants. Less governance proposal actions will need to be taken to restore additional liquidity to the markets. This method is less risky, however it fixes less of the underlying problem than the standard approach.

Our Experience With Moonwell

Solidity Labs helped Moonwell’s launch on base. This project included testing the MultiRewardsDistributor, uplifting the smart contracts to version 8 of Solidity, upgrading existing staking infrastructure, building governance tooling for cross-chain governance, and building the new TemporalGovernor Cross-chain governance primitive. We recently completed the xWELL token, which turns Moonwell into a native multichain token. All of these past experiences working together have given our team an in-depth understanding of both the current protocol contracts and the future direction of the Moonwell Protocol.

About Us

Solidity labs is a boutique consulting firm specializing in smart contract development and security. Our team is composed of engineers who have decades of blockchain and security experience. Smart contracts developed by our team have held billions in assets and have never been hacked. We have participated in and helped over 10 projects deliver their smart contract systems from idea to implementation. Additionally, we have helped create and audit four different bespoke lending protocols, giving our team an intimate understanding of the problem space and the ability to reason through the consequences of changes in complex systems like Moonwell. At Volt Protocol, we audited the Compound V2 codebase, and identified a critical vulnerability that was eventually patched.

The key insight that led to the founding of our firm is the realization that DeFi and smart contract projects increasingly need to shift left on security. Security doesn’t start once development is complete, it is a journey throughout the development process and requires developers to be vigilant and up to date on the latest exploits. To further that end and help projects shift left, we have built a custom proposal simulation framework that can be applied to enhance and ensure the security of these upgrades and offer continuous security reviews for our clients. This framework is currently being used to help secure Moonwell and makes governance proposal generation and simulation, fast and secure.

We would appreciate the support of the Moonwell community to vote for our team as we have invested significant energy into figuring out how to resolve this issue. We will apply the same rigor to executing these governance proposals as we did specifying them and will utilize every resource at our disposal to ensure this recovery is executed in a safe and secure manner.

4 Likes