Switch Moonwell Moonbeam's Markets to using API3’s OEV Enabled Data Feeds

Author: Dave Connor, API3

Related Discussions: none

Submission Date: 21/11/24

Summary

Moonwell is one of the highest revenue generating protocols in DeFi (https://defillama.com/fees?category=Lending), currently sitting at 11th in revenue over the last year with over $1.2m generated. This proposal seeks to boost the yield that Moonwell’s Moonbeam deployment generates by switching to using API3’s dAPIs (data feeds) for all markets. API3’s OEV data feeds have been used in production without problems for months by many lending markets, including Compound V2 forks (like Moonwell) and have just been integrated by Compound itself. API3’s OEV feeds can be switched to easily from contracts expecting a “push” oracle, like Chainlink, without the need to change any code, reducing risks.

This proposal deliberately limits the scope of the change to Moonwell on Moonbeam. This is intended to act as a proof of concept, and demonstrate the value that OEV can bring, with the potential to expand to other Moonwell deployments in future (subject to separate future proposals) to maximise the revenue to Moonwell. Similarly, while this proposal directs the revenue from OEV to the addReserves function of the GLMR core maket on Moonbeam, there are many alternatives that could be explored in future proposals, such reducing effective liquidation penalties. API3 will also develop a Dune dashboard to demonstrate the value that OEV is bringing to Moonwell in an easily accessible way for Moonwell users.

OEV solutions introduce complexities so this proposal will be as detailed as possible. Where possible, further resources are linked to, but questions are encouraged and welcomed where something is insufficiently well explained

Overview

To ensure any positions eligible for liquidation are promptly liquidated, reducing the risk of protocol level bad debt, Moonwell pays an incentive to whoever is able to trigger them (“searchers”). This process is open to everyone. Triggering a successful liquidation pays 7% of the liquidated position to whichever searcher was able to trigger it.

This setup for decentralising liquidations is common in defi, with almost every other lending market paying similar incentives to ensure reliable triggering of liquidations. As this is effectively a source of free money, it tends to be incredibly competitive. On various chains it is possible for searchers to bid for priority over other searchers, and quite often they are willing to pay a large percentage of what they expect to make as a reward for this - because making some money, even a small amount, is better than none if they are outbid.

Where the ability to bid for priority exists, searchers are happy with a much smaller amount in exchange for triggering liquidations than lending markets typically pay. From the point of view of the lending market, this can be considered wasted liquidity, as it is effectively not needed, and does not end up with whoever triggered the liquidation, who was happy receiving less.

Moonwell is a Compound V2 fork. Lending markets based on Compound V2 are built expecting “push” oracles. A push oracle can be described as an oracle that keeps an on-chain reference price updated, so that it can be used at any time by smart contracts on the same chain.

Price updates by push oracles like Chainlink and API3 are pushed on chain based on two criteria - time and deviation. Time is a set frequency of update, regardless of price movement. Deviation based updates allow the on chain price to vary by up to a set percentage of the real time price before an update is triggered. The actual data providers update their prices offchain at a far more granular level. When a Searcher sees one of these more current prices offchain that would trigger a liquidation onchain, they can bid for the right to pull that more current price on chain and to bundle a liquidation with it thus ensuring they get the associated rewards. These additional updates simply provide redundancy and granularity to the existing push updates.

API3’s OEV data feeds allow searchers to trigger additional data feed updates which in effect gives them a “fast lane” for liquidations, and gives those willing to pay priority over the other people competing. The searchers are unable to change data values, and can only trigger an earlier update (from the same data providers) than would otherwise occur. There is more information about how this works exactly here, but a brief summary is:

  • Searchers monitor positions on lending markets that can be liquidated at certain price points
  • Searchers also monitor the aggregated prices from API3’s Data providers.
  • When the providers show a price offchain that would trigger a liquidation, searchers are able to bid for the right to trigger an extra update. The winning bid goes to API3, who split this with the dapp that the searchers trigger the update for
  • The winning searcher gets a signed transaction that only they can use, which triggers a data feed update for a specific pair, eg ETH/USD
  • The searcher then bundles this price update transaction in a multicall with the liquidation transaction. As the update is signed, only the winning bidder can issue the update, giving them priority to trigger the liquidation.
  • API3’s dAPIs have a 15 second delay for deviation based updates to allow winning searchers time to update the feed. This ensures that searchers know they’re able to use the update to trigger the liquidation, which optimises the value they’re willing to bid.

API3 then gives 80% of these bid proceeds back to the dapp, and retains 20% to split with data providers. It can be expected that competition between searchers to win these auctions will have similar effects as the ability to bid for priority on mainnet had, and trend towards the total value of the liquidation incentive.

For Moonwell, users pay a 10% penalty when they are liquidated, with 3% going into a safety reserve to guard against bad debt. Fully using API3’s OEV feeds across all markets would mean that up to 5.6% (80% of 7%) of the total amount liquidated would be returned to Moonwell.

All of API3’s feeds have OEV functionality built in, and have never had a misreport or downtime on any feed since inception… Large lending markets like Compound, YEI Finance, Init Capital, Mendi, Ionic and Orbit have switched over. Multiple users are Compound V2 forks, similar to, Moonwell, further proving compatibility. API3 will assume the costs of operating the necessary data feeds, and of the integration itself where necessary.

Redundancy vs Dependency

These updates are simply more current prices from the same providers. The OEV Network serves as a layer of redundancy and ensures liquidations happen exactly when they should rather than delaying or waiting for time or deviation-based prices to hit. If the entire OEV Network went down, the regularly pushed prices would update as they currently do and Searchers participating would be able to trigger liquidations exactly as they do now, OEV just adds additional updates from the same sources and additional venues where searchers can compete.

Motivation

OEV represents a new source of income for Moonwell. API3’s OEV solution offers a market leading 80% of OEV back to Moonwell, in comparison to Redstone’s 50%, and has the added benefit of demonstrated production usage for many months. API3 will also develop a Dune dashboard to demonstrate the value that OEV is bringing to Moonwell in an easily accessible way for Moonwell users.

Implementation

API3 will ensure every feed currently used by Moonwell on Moonbeam is provided. Moonwell will switch from reading prices from Chainlink’s data feeds to API3’s dAPIs. There is a Quantstamp audited version of the current Chainlink adapter available. Integration is as simple as reading a new price feed that happens to update more frequently than your current one. API3 is happy to provide all necessary technical assistance for this switch.

OEV accrued will be distributed to the GLMR core market on Optimism’s reserves using the addReserve function. This process will be visible on chain and verifible by Moonwell community members

Voting

Yay - Moonwell will switch from using Chainlink to API3 on Moonbeam. API3 will distribute 80% of the OEV proceeds to Moonwell, via the addReserves function on the GLMR core market on Moonbeam

Nay - No changes will be made

2 Likes

Thanks for the proposal. I have looked into the API3 OEV Compound proposal and have identified some risks with the integration that I will summarize below. Many of these concerns were also brought up by Gauntlet and Wintermute Governance in the API3 OEV proposal for Compound.

Added bad debt risk from data feed delays: Even if API3 sequencer does not suffer any issues, the data feeds used for opening and closing positions are delayed by 15-30 seconds. During long tail events where tokens experience significant volatility this could create a bad debt attack vector similar to the one experienced by Compound V2 during the price spike of UNI in February 2024. Arbitrageurs can create bad debt by exploiting the price discrepancy and this is not prevented by API3 OEV auctions as this attack can be atomically exploited. In addition to this, if the API3 sequencer suffers downtime the bad debt risks further increase as liquidations would now too be delayed by 15-30 seconds. To account for this risk, lending protocols should allow a lower LTV than they otherwise would have without the delay. I would like to see data that takes into account long tail events such as this and evaluates the added risks, and proposes changes that should be made to LTV offered by Moonwell protocol.

Added bad debt risk to the protocol from exclusive access to liquidations: API3 OEV auctions award exclusive access to liquidations for 60 seconds to a single winning bidder. This introduces the risk of a single actor not having the correct strategy or having having bugs that cause them to not capture every single liquidation during their exclusive 60s period, causing some liquidations to be delayed. This is a known risk that has been brought up with the Timeboost MEV auction for Arbitrum. Arbitrum anticipate a PBS style market developing around Timeboost where the winner can resell MEV opportunities to mitigate such risks. Such a re-selling market structure does not necessarily seem possible for API3 OEV auctions, and also requires a very developed MEV ecosystem to develop, potentially leaving the API3 mechanism with this risk.

Hurdles to onboard liquidators: There are large switching costs for liquidators to onboard to API3 because they must bridge funds to the API3 L2 and develop a bidding strategy based on efficiently using locked collateral.

Poor capital efficiency for liquidators: Liquidators cannot simply use flashloans with API3 liquidations, they have to lock up a portion of their bids on an L2 and suffer opportunity costs for their capital as well as liquidity constraints due to bridging time. Requiring upfront payment of a portion of the bid will lead to lower value capture and lower liquidator adoption especially due to the cyclical nature of liquidation opportunities. It is unlikely liquidators will want to keep significant funds on the API3 L2, and will be underfunded when big liquidation spikes occur. This leads to liquidators not having enough capital to bid up to 80-90% of the value on big opportunities like they could with flash loans alone.

Added risk to the liquidators: If the price changes significantly between when the liquidator places their bid on the API3 L2 and when they are awarded the data feed update, then they may suffer losses which complicates the liquidators bidding strategy.

here are the links to the concerns expressed by Gauntlet and Wintermute Governance:

Just replying in here for anyone watching to point out that we’ve altered the proposal from Optimism to Moonbeam. This change would allow API3’s OEV to be directly compared to the Solidity Labs proposal, as well as act as a proof of concept on a smaller market. We’ll answer the other points raised shortly :slight_smile:

Hey llama, Sorry for the delay answering here, wanted to get some feedback from the technical team to answer. Forwarding it below:

Added bad debt risk from data feed delays: Even if API3 sequencer does not suffer any issues, the data feeds used for opening and closing positions are delayed by 15-30 seconds. During long tail events where tokens experience significant volatility this could create a bad debt attack vector similar to the one experienced by Compound V2 during the price spike of UNI in February 2024. Arbitrageurs can create bad debt by exploiting the price discrepancy and this is not prevented by API3 OEV auctions as this attack can be atomically exploited. In addition to this, if the API3 sequencer suffers downtime the bad debt risks further increase as liquidations would now too be delayed by 15-30 seconds. To account for this risk, lending protocols should allow a lower LTV than they otherwise would have without the delay. I would like to see data that takes into account long tail events such as this and evaluates the added risks, and proposes changes that should be made to LTV offered by Moonwell protocol.

The compound V2 UNI v2 tweet you’ve listed mentions that this bad debt was the result of the TWAP anchor and not the Chainlink price feed failing to push the price in time. In the case of Uniswap the anchor period is 1800 seconds or 30 mins, which is significantly higher than 30seconds. A 30 minute twap could be problematic which is why we suggest protocols to not rely on TWAP feeds.
In our analysis of on-chain oracle data on Binance chain for BTC, BNB and wBTC since 2021, we couldn’t observe any instances where a 30second delay would have any impact on the performance of any lending markets on those chains. We will share the complete data soon, but just to give some perspective, the maximum deviation in a 60 second window from 2021 to present day has only been 3.6% for btc and this only happened twice. The other high deviations are less than 2%. We would not suggest any changes in LTVs are required. Additionally, if a rapid movement is likely to cause many liquidations, API3’s OEV model allows searchers to trigger extra updates to capture these, leading to more granular feeds, and immediate updates.

Added bad debt risk to the protocol from exclusive access to liquidations: API3 OEV auctions award exclusive access to liquidations for 60 seconds to a single winning bidder. This introduces the risk of a single actor not having the correct strategy or having having bugs that cause them to not capture every single liquidation during their exclusive 60s period, causing some liquidations to be delayed. This is a known risk that has been brought up with the Timeboost MEV auction for Arbitrum. Arbitrum anticipate a PBS style market developing around Timeboost where the winner can resell MEV opportunities to mitigate such risks. Such a re-selling market structure does not necessarily seem possible for API3 OEV auctions, and also requires a very developed MEV ecosystem to develop, potentially leaving the API3 mechanism with this risk.

This is incorrect, OEV auctions are currently divided into two phases, bid phase and award phase. each configurable to the market and feed. At the moment we are targeting a bid and award phase of 15 seconds each, which is why the delay on the base feed is 30 seconds. As you’ve stated arbitrum timeboost is also looking to give the highest bidder 1 minute advantage in the express lane and many suspect that this could create secondary markets where the 1 minutes window is resold in chunks to the searchers thereby maximizing revenue for the Arbtirum DAO, this can also easily happen with OEV. In the OEV award phase, the auction winner can resell the opportunity in a secondary market since the updater address can be tied to a contract that can only allow the winner of the secondary marketplace auction to perform the OEV update. As with timeboost, its a possibility but as you can see its very doable in theory.

Hurdles to onboard liquidators: There are large switching costs for liquidators to onboard to API3 because they must bridge funds to the API3 L2 and develop a bidding strategy based on efficiently using locked collateral.

The collateral only gets locked upon winning a bid and gets released upon executing the price update. This means that with a very small amount of collateral, a user can place very large amount of bids with large bids amount. The collateral is only 5% of the bid amount and gets released upon submitting fulfilment of price update. While we suspect this might lead to initial friction in onboarding, the slashing mechanism disincentivizes searchers from placing bids that they don’t follow through with ensuring that updates arrive on-chain as soon as they are required. The 5% number can be tweaked over time should it be necessary to ensure optimal performance vs searcher QOL .

Poor capital efficiency for liquidators: Liquidators cannot simply use flashloans with API3 liquidations, they have to lock up a portion of their bids on an L2 and suffer opportunity costs for their capital as well as liquidity constraints due to bridging time. Requiring upfront payment of a portion of the bid will lead to lower value capture and lower liquidator adoption especially due to the cyclical nature of liquidation opportunities. It is unlikely liquidators will want to keep significant funds on the API3 L2, and will be underfunded when big liquidation spikes occur. This leads to liquidators not having enough capital to bid up to 80-90% of the value on big opportunities like they could with flash loans alone.

Flashloans to pay winning bids are possible with API3, see here - OEV Searching | Documentation.

1 Like