I built a Neo4j Graph DB To Show NFT Data
AKA ... The love-child of Neo4j and the Alchemy NFT API.
I just picked up a month-long consulting gig using Neo4j, something I have zero experience with. Fortunately the job doesn't start for a week or two, and this weekend I had lots of free time.
Neo4j is a graph database. Data is represented by Nodes and Relationships. I've worked with relational databases a lot, pretty much that's my only experience storing data for commercial projects. In a relational db, data is stored in tables and then relationships are inferred between them at query time. In a graph db, relationships are stored as stored as pointers between nodes . Relational databases are great for storing lots and lots of information, graph databases shine with projects where you need to infer relationships between data. Data is useless without analysis, and a good data scientist can probably do amazing work with just about any dataset. I'm not a data scientist, and I am a very visual person. As I dug into graph databases, I started to wonder if graph visualization would help me see patterns I might not normally notice.
Imagine a simple example modeling blockchain wallets and the NFT contracts they own. One wallet owns many NFTs and one NFT contract is owned by many wallets, a common many-to-many relationship. In a graph model, we create a schema with two Nodes (Wallet and Contract) and one Relationship between them (OWNS). In relational mode, we create three tables, a Wallet table, a Contract table and a many-to-many join table. The actual relationship between Wallets and Contracts isn't stored, it's generated at query time.
There's probably a lot more to graph databases, but after a weekend of learning, that's about what fits in my brain.
I don't know a lot about the NFT space. From a technical perspective, I know the ERC721 standard, I built my generative music system interdimensional.one right on top of it. I've read a lot about financial NFTs and soul-bound NFTs. I've earned PoK NFTs from The Road To Web3 and from Patrick Collins's 32-hour course. But my understanding of the space is purely technical, I don't know much about investing in the space, I don't know much about how to find up and coming projects.
So ... enter my weekend project. I started wondering what would happen if I:
- Started with popular NFT contracts.
- Looked at their owners.
- Looked at the top 20 NFTs owned by them.
In looking at the NFTs collected by people who popular collections like BAYC and CyptoPunks would I see patterns emerge? Would I find smaller collections with lower floor prices that haven't popped yet, but might in the near future?
I've been wanting to find a reason to play around with the new Alchemy NFT API too, so I figured this was a great way to kill two birds with one stone.
Ok, now for the fun part. Y'all can find all my code for this in GitHub.
npm init -y
npm install alchemy-sdk ethers neo4j-driver
I added type:module to my package.json too as I'm using ES6-style modules. Here's my final package.json file.
{
"name": "neo4jnft",
"version": "1.0.0",
"description": "Weekend project using Neo4j to browse data from AlchemyNFT API",
"main": "import_from_contract_address.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "Luke Cassady-Dorion",
"license": "MIT",
"type": "module",
"dependencies": {
"alchemy-sdk": "^2.0.3",
"ethers": "^5.7.0",
"neo4j-driver": "^4.4.7"
}
}
Before you go any further, you'll need to create a free account with Alchemy and create an app on the Ethereum Mainnet. Do that, grab your API key and save it somewhere.
Next you'll need access to a Neo4j database, I built using the free Aura Cloud option, but you could probably build using the desktop version too. Create a new instance, and make note of your id, password and connection URI.
Ok, cool, cool, cool. With everything setup, now we can write a small node.js application to query the Alchemy NFT API and move that data over to Neo4j. It's going to be a short project, just around 200 lines of code or so. Let's start with a framework setting up out main() function and all helper functions we need. Notice we're using async functions as we'll be making calls to the blockchain and want to pause execution while the queries run.
import { Network, Alchemy } from "alchemy-sdk";
import neo4j from "neo4j-driver";
import ethers from "ethers";
/**
* @notice Adds a Contract node with the specified address
* @param {*} alchemySession Alchemy Session
* @param {*} neo4jSession Neo4J Session
* @param {*} address Wallet address to add
* @returns true if successful, false if command fails or if the Contract already exists
*/
async function addContract(alchemySession, neo4jSession, address) {
}
/**
* @notice Checks if a Contract node exists with the specific address.
* @return True if exists, false otherwise
* @param {*} neo4jSession Neo4J Session
* @param {*} address Contract address to check
*/
async function contractExists(neo4jSession, address) {
}
/**
* @notice Adds a Wallet node with the specified address
* @param {*} neo4jSession Neo4J Session
* @param {*} address Wallet address to add
* @returns true if successful, false if command fails or if the wallet already exists
*/
async function addWallet(neo4jSession, address) {
}
/**
* @notice Checks if a Wallet node exists with the specific address.
* @return True if exists, false otherwise
* @param {*} neo4jSession Neo4J Session
* @param {*} address Contract address to check
*/
async function walletExists(neo4jSession, address) {
}
/**
* @notice Adds an OWNS relationship between the WALLET and CONTRACT.
* If the address already exists, a duplicate is NOT added.
* @param {*} neo4jSession Neo4j session
* @param {*} contractAddress NFT contract address owned by wallet
* @param {*} walletAddress Wallet address
* @returns True if successful, false if either address does not exist
*/
async function addRelationship(neo4jSession, contractAddress, walletAddress) {
}
async function main() {
}
main()
.then(() => process.exit(0))
.catch((error) => {
console.error(error);
process.exit(1);
});
First thing we need to do in main() is setup our link to the Alchemy NFT API and Neo4j. It's probably good practice to pull out your API key and DB password into an .env file, but I wanted to make this all really easy to read so I didn't bother.
I convert all addresses to their checksum version using ethers as Neo4j defaults to case-sensitive searches and I wanted to normalize address case before saving.
let mainNFTContract = "0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D"; // the NFT contract address to search on (BAYC)
mainNFTContract = ethers.utils.getAddress(mainNFTContract); // Checksum version
// Alchemy config
const alchemySettings = {
apiKey: "", // Replace with your Alchemy API Key.
network: Network.ETH_MAINNET, // Replace with your network identifier
};
// Neo4J Config
const neo4JSettings = {
uri: "", // Replace with your Neo4j URI
user: "neo4j", // Replace with your Neo4j user
password: "", // Replace with your Neo4j password
};
// Neo4J Driver
const driver = neo4j.driver(
neo4JSettings.uri,
neo4j.auth.basic(neo4JSettings.user, neo4JSettings.password),
);
// Neo4J Session
const neo4JSession = driver.session({ database: "neo4j" });
// Alchemy SDK
const alchemySession = new Alchemy(alchemySettings);
Before we go any further, let's take a detour and look at Cypher, the Neo4j query language. It probably goes without saying that I am not an expert in this stuff, I've only put in a few hours learning so far. But ... we're not doing anything super complex just yet, so it doesn't really matter.
// create wallet
CREATE (:Wallet {address: '0xE9AF6D5309Dfc433eA5eA0d1EA610B7448f2185d'})
// create contract
CREATE (:Contract {title: 'The Ape List Official', symbol: 'TAL', address: '0x2f07ab5e3f349a4803a3805049ed57ce26302cb5', totalSupply: '4276'})
// add a relationship, but don't duplicate
MATCH (w), (c) WHERE w.address = "0xE9AF6D5309Dfc433eA5eA0d1EA610B7448f2185d" AND c.address= "0x2f07ab5e3f349a4803a3805049ed57ce26302cb5"
MERGE (w)-[:OWNS]->(c);
The first two commands create new nodes using the CREATE command. CREATE (:Wallet {address:foo ''}) Creates a new Wallet node with address attribute set to "foo". The second command creates a Contract node with a bunch more attributes. What's cool is unlike relational models, in a graph database new attributes can be easily added whenever. Right now I'm not tracking floor price in the NFT contract, but could just add that in whenever without having to change the schema (as I would with a relational database.)
The final command uses a MATCH command to identify one wallet and one contract and then creates an OWNS relationship between them. By using MERGE, we ensure duplicates are never created. Relationships can also have attributes. I didn't do it here as I wanted to keep things simple, but I could modify OWNS to show the case where one wallet owns more than one NFT from a collection.
To make my main() function simple and easy to read, I abstracted out all database interactions into separate functions. From main we call either addContract(), addWallet() or addRelationship() passing along addresses and then the function takes over. The addFoo() functions first check for if the node already exists (a common case as we're looking to identify NFT contracts owned by multiple people) and then adds it if it doesn't.
/**
* @notice Adds a Contract node with the specified address
* @param {*} alchemySession Alchemy Session
* @param {*} neo4jSession Neo4J Session
* @param {*} address Wallet address to add
* @returns true if successful, false if command fails or if the Contract already exists
*/
async function addContract(alchemySession, neo4jSession, address) {
try {
// check if already exists
if (await contractExists(neo4jSession, address)) return false;
// doesn't exist, so add it
// 1. get contract metadata
// 2. use metadata to create our initial CONTRACT node representing the main contract
const contractMetadata = await alchemySession.nft.getContractMetadata(address);
const command = `CREATE (:Contract {title: '${contractMetadata.name}', symbol: '${contractMetadata.symbol}', address: '${address}', totalSupply: '${contractMetadata.totalSupply}'})`;
console.log(command);
const result = await neo4jSession.run(command);
return true;
} catch (error) {
console.error(error);
return false;
}
}
/**
* @notice Checks if a Contract node exists with the specific address.
* @return True if exists, false otherwise
* @param {*} neo4jSession Neo4J Session
* @param {*} address Contract address to check
*/
async function contractExists(neo4jSession, address) {
try {
const command = `MATCH (c:Contract) WHERE c.address = '${address}' RETURN c `;
const result = await neo4jSession.run(command);
return result.records.length >= 1;
} catch (error) {
console.error(error);
return false;
}
}
/**
* @notice Adds a Wallet node with the specified address
* @param {*} neo4jSession Neo4J Session
* @param {*} address Wallet address to add
* @returns true if successful, false if command fails or if the wallet already exists
*/
async function addWallet(neo4jSession, address) {
try {
// check if already exists
if (await walletExists(neo4jSession, address)) return false;
// doesn't exist, so add it
const command = `CREATE (:Wallet {address: '${address}'})`;
console.log(command);
const result = await neo4jSession.run(command);
return true;
} catch (error) {
console.error(error);
return false;
}
}
/**
* @notice Checks if a Wallet node exists with the specific address.
* @return True if exists, false otherwise
* @param {*} neo4jSession Neo4j Session
* @param {*} address Contract address to check
*/
async function walletExists(neo4jSession, address) {
try {
const command = `MATCH (w:Wallet) WHERE w.address = '${address}' RETURN w `;
const result = await neo4jSession.run(command);
return result.records.length >= 1;
} catch (error) {
console.error(error);
return false;
}
}
/**
* @notice Adds an OWNS relationship between the WALLET and CONTRACT.
* If the address already exists, a duplicate is NOT added.
* @param {*} neo4jSession Neo4j session
* @param {*} contractAddress NFT contract address owned by wallet
* @param {*} walletAddress Wallet address
* @returns True if successful, false if either address does not exist
*/
async function addRelationship(neo4jSession, contractAddress, walletAddress) {
try {
// make sure the wallet and contract exist
if (!(await contractExists(neo4jSession, contractAddress))) return false;
if (!(await walletExists(neo4jSession, walletAddress))) return false;
// they exist, so create a relationship
const command = `MATCH (w), (c) WHERE w.address = "${walletAddress}" AND c.address= "${contractAddress}" MERGE (w)-[:OWNS]->(c);`;
console.log(command);
const result = await neo4jSession.run(command);
return true;
} catch (error) {
console.error(error);
return false;
}
}
The main() function is where I use the Alchemy NFT API to query data and pass it to the above functions to store in our database. I gotta say, the Alchemy API is so easy to use (and read), there's not much I can say about it. We just
- Create a session
const alchemySession = new Alchemy(alchemySettings); - Query the API to get a contract's owners
await alchemySession.nft.getOwnersForContract(mainNFTContract); - Query the API to get NFTs owned by a wallet
await alchemySession.nft.getNftsForOwner(mainNFTContract);
Here's the function in full.
async function main() {
let mainNFTContract = "0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D"; // BAYC ERC-721 (Ethereum)
// let mainNFTContract = "0xb47e3cd837ddf8e4c57f05d70ab865de6e193bbb"; // CryptoPunks Custom Contract (Ethereum)
//let mainNFTContract = "0x60576a64851c5b42e8c57e3e4a5cf3cf4eeb2ed6"; // MintKudos ERC-721 (Polygon)
mainNFTContract = ethers.utils.getAddress(mainNFTContract); // Checksum version
// Alchemy config ETH
const alchemySettingsETH = {
apiKey: "", // Replace with your Alchemy API Key.
network: Network.ETH_MAINNET, // Replace with your network identifier
};
// Alchemy config Polygon
const alchemySettingsPolygon = {
apiKey: "", // Replace with your Alchemy API Key.
network: Network.MATIC_MAINNET, // Replace with your network identifier
};
// Neo4J Config
const neo4JSettings = {
uri: "", // Replace with your Neo4j URI
user: "", // Replace with your Neo4j user
password: "", // Replace with your Neo4j password
};
// Neo4J Driver
const driver = neo4j.driver(
neo4JSettings.uri,
neo4j.auth.basic(neo4JSettings.user, neo4JSettings.password),
);
// Neo4J Session
const neo4JSession = driver.session({ database: "neo4j" });
// Alchemy SDK
const alchemySessionEth = new Alchemy(alchemySettingsETH);
const alchemySessionPolygon = new Alchemy(alchemySettingsPolygon);
// add the main contract
await addContract(alchemySettingsETH, neo4JSession, mainNFTContract);
// 1. get all owners for that contract and add them as WALLET nodes
// 2. create an OWNS relationship between WALLETs -> CONTRACT
const ownersForContract = await alchemySessionEth.nft.getOwnersForContract(mainNFTContract);
const owners = ownersForContract.owners;
// I decided to limit things to just the first 500 owners as it was too hard to
// render more than about 10000 nodes.
console.log("owner count=", owners.length);
let ownerCount = owners.length;
if (ownerCount > 500) ownerCount = 500;
for (let i = 0; i <= ownerCount; i++) {
console.log(`${i}:${ownerCount} adding owner`);
const walletAddress = ethers.utils.getAddress(owners[i]);
await addWallet(neo4JSession, walletAddress);
await addRelationship(neo4JSession, mainNFTContract, walletAddress);
}
//3. again iterate over the owners, this time add in all contracts owned by that owner
for (let i = 0; i <= ownerCount; i++) {
try {
console.log(`${i}:${ownerCount} getting NFTS owned by ${owners[i]}`);
// Again I'm limiting the amount of data so I can render it properly
const nftsOwnedByWallet = await alchemySessionEth.nft.getNftsForOwner(owners[i]);
let nftOwnedCount = nftsOwnedByWallet.ownedNfts.length;
if (nftOwnedCount > 20) nftOwnedCount = 20;
for (let j = 0; j <= nftOwnedCount; j++) {
const contractAddress = ethers.utils.getAddress(
nftsOwnedByWallet.ownedNfts[j].contract.address,
);
const walletAddress = ethers.utils.getAddress(owners[i]);
await addContract(alchemySessionEth, neo4JSession, contractAddress);
await addRelationship(neo4JSession, contractAddress, walletAddress);
}
} catch (error) {
console.error(error);
}
}
// clean up
await driver.close();
await neo4JSession.close();
console.log("i'm done");
}
Ok, so what did I find?
The BAYC contract has over 6000 owners. On my first attempt I pulled in all owners and the top 100 NFTs owned by them. Right away, I found while I could query the data, my computer couldn't render more than 8-10K nodes. So I went back and put in some limits. This time just taking the first 500 wallets and a max of 20 NFTs owned by each.
BAYC is a really popular collection, lots of celebs seem to dig it and their parties get the tech press excited. Given its popularity, it's not surprising to see such a dense graph. BAYC holders also hold lots of other NFTs. In many cases, a single wallet holds NFTs not held by others ... but there are some nice outliers where we have less than 100 holders, maybe indicating the collection is up and coming.
This next one looks at the CryptoPunks contract. CryptoPunks is an older custom contract that predates ERC-721, so it was a good example to try with with Alchemy API. Very cool the same API is able to deliver information from different contract standards.
CryptoPunks is popular with developers and hardcore collectors, not necessarily the celeb audience you get with BAYC. Again, we see the same pattern of lots of NFTs only owned by a wallet or two, but then a handful of NFT contracts that seem to have a good number of collectors within the Punks community.
With the last design I was able to take advantage of how the Alchemy NFT API is cross-platform by looking at the MintKudos contract. MintKudos creates the PoK NFTs we all earn by completing Road To Web 3 challenges, I was curious what other NFTs the community owns (is that me being nosy?:)).
MintKudos NFTs are released on Polygon, to get a list of owners I need to query on that Polygon. When querying getNftsForOwner() I run that on Ethereum using the wallet address I discovered from calling getOwnersForContract() on Polygon.
With this case we're looking mostly at wallets of people learning to be developers, not NFT collectors. Makes sense the graph is much less dense, meaning overall fewer NFTs are owned. As with CryptoPunks, you get a similar pattern of many NFTs owned uniquely by only one wallet. On the right side of the screen you see NFT collections owned by smaller numbers of wallets.
Ok, that's it for now. My next project is to really dig into Cyper as a query language ... but I think I want to try to hurry up and finish that damn 32-hour Solidity course first.
Here's the full code listing. Or go grab it from GitHub.
import { Network, Alchemy } from "alchemy-sdk";
import neo4j from "neo4j-driver";
import fs from "fs-extra";
import ethers from "ethers";
/**
* @notice Adds a Contract node with the specified address
* @param {*} alchemySession Alchemy Session
* @param {*} neo4jSession Neo4J Session
* @param {*} address Wallet address to add
* @returns true if successful, false if command fails or if the Contract already exists
*/
async function addContract(alchemySession, neo4jSession, address) {
try {
// check if already exists
if (await contractExists(neo4jSession, address)) return false;
// doesn't exist, so add it
// 1. get contract metadata
// 2. use metadata to create our initial CONTRACT node representing the main contract
const contractMetadata = await alchemySession.nft.getContractMetadata(address);
const command = `CREATE (:Contract {title: '${contractMetadata.name}', symbol: '${contractMetadata.symbol}', address: '${address}', totalSupply: '${contractMetadata.totalSupply}'})`;
console.log(command);
const result = await neo4jSession.run(command);
return true;
} catch (error) {
console.error(error);
return false;
}
}
/**
* @notice Checks if a Contract node exists with the specific address.
* @return True if exists, false otherwise
* @param {*} neo4jSession Neo4J Session
* @param {*} address Contract address to check
*/
async function contractExists(neo4jSession, address) {
try {
const command = `MATCH (c:Contract) WHERE c.address = '${address}' RETURN c `;
const result = await neo4jSession.run(command);
return result.records.length >= 1;
} catch (error) {
console.error(error);
return false;
}
}
/**
* @notice Adds a Wallet node with the specified address
* @param {*} neo4jSession Neo4J Session
* @param {*} address Wallet address to add
* @returns true if successful, false if command fails or if the wallet already exists
*/
async function addWallet(neo4jSession, address) {
try {
// check if already exists
if (await walletExists(neo4jSession, address)) return false;
// doesn't exist, so add it
const command = `CREATE (:Wallet {address: '${address}'})`;
console.log(command);
const result = await neo4jSession.run(command);
return true;
} catch (error) {
console.error(error);
return false;
}
}
/**
* @notice Checks if a Wallet node exists with the specific address.
* @return True if exists, false otherwise
* @param {*} neo4jSession Neo4j Session
* @param {*} address Contract address to check
*/
async function walletExists(neo4jSession, address) {
try {
const command = `MATCH (w:Wallet) WHERE w.address = '${address}' RETURN w `;
const result = await neo4jSession.run(command);
return result.records.length >= 1;
} catch (error) {
console.error(error);
return false;
}
}
/**
* @notice Adds an OWNS relationship between the WALLET and CONTRACT.
* If the address already exists, a duplicate is NOT added.
* @param {*} neo4jSession Neo4j session
* @param {*} contractAddress NFT contract address owned by wallet
* @param {*} walletAddress Wallet address
* @returns True if successful, false if either address does not exist
*/
async function addRelationship(neo4jSession, contractAddress, walletAddress) {
try {
// make sure the wallet and contract exist
if (!(await contractExists(neo4jSession, contractAddress))) return false;
if (!(await walletExists(neo4jSession, walletAddress))) return false;
// they exist, so create a relationship
const command = `MATCH (w), (c) WHERE w.address = "${walletAddress}" AND c.address= "${contractAddress}" MERGE (w)-[:OWNS]->(c);`;
console.log(command);
const result = await neo4jSession.run(command);
return true;
} catch (error) {
console.error(error);
return false;
}
}
async function main() {
let mainNFTContract = "0xBC4CA0EdA7647A8aB7C2061c2E118A18a936f13D"; // BAYC ERC-721 (Ethereum)
// let mainNFTContract = "0xb47e3cd837ddf8e4c57f05d70ab865de6e193bbb"; // CryptoPunks Custom Contract (Ethereum)
// let mainNFTContract = "0x60576a64851c5b42e8c57e3e4a5cf3cf4eeb2ed6"; // MintKudos ERC-721 (Polygon)
mainNFTContract = ethers.utils.getAddress(mainNFTContract); // Checksum version
// Alchemy config ETH
const alchemySettingsETH = {
apiKey: "", // Replace with your Alchemy API Key.
network: Network.ETH_MAINNET, // Replace with your network identifier
};
// Alchemy config Polygon
const alchemySettingsPolygon = {
apiKey: "", // Replace with your Alchemy API Key.
network: Network.MATIC_MAINNET, // Replace with your network identifier
};
// Neo4J Config
const neo4JSettings = {
uri: "", // Replace with your Neo4j URI
user: "", // Replace with your Neo4j user
password: "", // Replace with your Neo4j password
};
// Neo4J Driver
const driver = neo4j.driver(
neo4JSettings.uri,
neo4j.auth.basic(neo4JSettings.user, neo4JSettings.password),
);
// Neo4J Session
const neo4JSession = driver.session({ database: "neo4j" });
// Alchemy SDK
const alchemySessionEth = new Alchemy(alchemySettingsETH);
const alchemySessionPolygon = new Alchemy(alchemySettingsPolygon);
// add the main contract
await addContract(alchemySettingsETH, neo4JSession, mainNFTContract);
// 1. get all owners for that contract and add them as WALLET nodes
// 2. create an OWNS relationship between WALLETs -> CONTRACT
const ownersForContract = await alchemySessionEth.nft.getOwnersForContract(mainNFTContract);
const owners = ownersForContract.owners;
// I decided to limit things to just the first 500 owners as it was too hard to
// render more than about 10000 nodes.
console.log("owner count=", owners.length);
let ownerCount = owners.length;
if (ownerCount > 500) ownerCount = 500;
for (let i = 0; i <= ownerCount; i++) {
console.log(`${i}:${ownerCount} adding owner`);
const walletAddress = ethers.utils.getAddress(owners[i]);
await addWallet(neo4JSession, walletAddress);
await addRelationship(neo4JSession, mainNFTContract, walletAddress);
}
//3. again iterate over the owners, this time add in all contracts owned by that owner
for (let i = 0; i <= ownerCount; i++) {
try {
console.log(`${i}:${ownerCount} getting NFTS owned by ${owners[i]}`);
// Again I'm limiting the amount of data so I can render it properly
const nftsOwnedByWallet = await alchemySessionEth.nft.getNftsForOwner(owners[i]);
let nftOwnedCount = nftsOwnedByWallet.ownedNfts.length;
if (nftOwnedCount > 20) nftOwnedCount = 20;
for (let j = 0; j <= nftOwnedCount; j++) {
const contractAddress = ethers.utils.getAddress(
nftsOwnedByWallet.ownedNfts[j].contract.address,
);
const walletAddress = ethers.utils.getAddress(owners[i]);
await addContract(alchemySessionEth, neo4JSession, contractAddress);
await addRelationship(neo4JSession, contractAddress, walletAddress);
}
} catch (error) {
console.error(error);
}
}
// clean up
await driver.close();
await neo4JSession.close();
console.log("i'm done");
}
main()
.then(() => process.exit(0))
.catch((error) => {
console.error(error);
process.exit(1);
});