A User-Centric Evaluation of Smart Contract Analysis Tools in Decentralised Finance (DeFi)

Blockchain and smart contract technology have led to the creation of an alternative financial system called Decentralised Finance (DeFi) which has grown exponentially in the last year alone to a current value of $76B. Without a central custodian or regulator, non-technical users may find it difficult to assess the security of their favourite projects. In this trustless environment, can the current state-of-the-art smart contract analysis tools be used by non-technical users to protect investors from incurring losses and improving the security in the space? In the paper, we review the literature focusing on well-known vulnerabilities of financial smart contracts and show the scale of successful DeFi attacks. By analysing the root cause of recent exploits of contracts, we assess the feasibility of detecting these vulnerabilities by automatic verification. We investigate 21 analysis tools for detecting vulnerabilities in smart contracts with an in-depth evaluation of six tools: Slither, Mythril, DerScanner, Manticore, Oyente and Securify v2. The tools were evaluated for their efficiency and accuracy against a custom dataset containing 28 vulnerable and 16 healthy smart contracts and are ultimately rated based on how useful they may be from a DeFi user perspective. The results indicate that, while Slither received the highest rating, none of the existing tools can successfully assist DeFi users at present due to lack of reliability or lack of simplicity for the targeted market.