I Built a Node.js Script To Upload Files To The Bundlr Permaweb

One (of the many) features I didn't have time to implement in Interdimensional.One before sending to the Polygon Hackathon is storage of all NFT assets on the permaweb.

You can read a full breakdown of the architecture I built for Interdimensional.One, or if you just need a refresher, here's the TLDR. I built a dApp to create generative music and art pieces using color and sound design information that comes from NFTs. An NFT prototype exists as a color and also ~5 MP3 files containing sound design information. Each prototype can be minted n times, currently set to 42. As it's built now, the color is stored on-chain along with a JSON file giving the location of the MP3 files. As each MP3 file is 100-700K, I didn't think storing them on-chain was an economically sound idea (not even sure it's possible?). In the version I submitted to the Hackathon, I served the files from the same Vercel server hosting the website.

I knew at the time that storing the files on just one random web server wasn't going to cut it for the final release, but ... you know ... sometimes deadlines come before everything is done and decisions have to be made.

With virtually all NFT projects, the actual art (pic, music, video, html...) is not stored on-chain. It's stored in a permaweb product like Bundlr (Arweave), Filecoin, IPFS or even times just on a random web server. Permaweb solutions guarantee a URL will always point to the same thing. If you buy an NFT of a rainbow, and that rainbow pic is stored on the permaweb, it is impossible for someone to go in and change the pic at the end of the URL to something else. It will be rainbows forever and ever.

While it's not possible to change what's at the end of the URL, not all permaweb solutions guarantee the file will always be there. With IPFS it's very possible your data will be deleted eventually (you can pin it with services like pinata), however with Bundlr and Arweave you're guaranteed to have access to it for ~200 years.

With Arweave you pay using its currency AR to upload a file and then you get a link that just works forever. Arweave sells storage in chunks of 256K, which means a 50K file costs the same to store as a 250K file. Bundlr is built on top of Arweave and adds a few nice features. Instead of just AR, they accept payment in most popular crypto currencies (ETH, AVAX, LINK, FTM, SOL, MATIC, others). Bundlr can be much more cost effective too as you only pay for the storage you use. Smaller files are bundled up together and then stored on Arweave.

Ok, cool, cool, cool. Let's dig in and write some code. We'll create two node.js scripts, one to upload a series of files to Bundlr and another to print out the price to store files of different sizes.

Setup a project

mkdir bundlr-uploader
cd bundlr-uploader
npm init -y
npm install @bundlr-network/client
code .

I started out with this example over on the Bundlr site. If your experience is anything like mine, you'll copy and paste the code and then bang your head against the wall for an hour trying to figure out why it doesn't run. The example on the website starts out creating a new Bundlr client object like this const bundlr = new Bundlr("https://node1.bundlr.network", "arweave", key), however that doesn't work. Gives an error saying TypeError: Bundlr is not a constructor. I probably googled the error 10 different ways before I thought to check their GitHub page, where I found this post telling me an alternative way const bundlr = new Bundlr.default("https://node1.bundlr.network", "arweave", key). I don't know why I always forget to check GitHub directly, I guess I just assume Google does a good job indexing it ... which apparently isn't true.

Ok, I wanted to call that out specifically in case you're trying it at home and get the same problem.

This script is really short, so here it is all at once. It uses the Bundlr Devnet to upload files using testnet MATIC to pay. Using the Devnet is a great way to test things out without using real money, however the files are only stored for ~7 days. In my example, I'm using a wallet funded with MATIC from the Mumbai testnet. The private key I supply when creating a new Bundlr object points to the wallet holding the testnet MATIC.

You can use any of their supported currencies, get some from a testnet faucet before trying to run the script.

This code is about half mine, half from the Bundlr site.

// import the client
import Bundlr from "@bundlr-network/client";
import fs from "fs";
import dotenv from "dotenv";
dotenv.config();

/**
 * @notice script to upload a series of files to the Bundlr network
 * Currently configured to use the devnet. It's free to use the Devnet,
 * but files are only archived for a 7 days.
 */
async function main() {
    // initialise a bundlr client
    // NOTE: The online example (https://docs.bundlr.network/docs/client/examples/full-example)
    // uses the following: bundlr = new Bundlr(...) which throws an error.
    // To run properly from node, you need to change to the version I have below new Bundlr.default().
    // This is documented here https://github.com/Bundlr-Network/js-sdk/issues/50
    const bundlr = new Bundlr.default("https://devnet.bundlr.network", "matic", process.env.PRIVATE_KEY, {
        providerUrl: process.env.MUMBAI_RPC,
    });

    // get account balance
    const balance = await bundlr.getLoadedBalance();
    console.log("account balance=", balance.toString());

    // convert it into decimal units
    const decimalBalance = bundlr.utils.unitConverter(balance);
    console.log("decimalBalance=", decimalBalance.toString());

    // List all file names to upload here
    const filesToUpload = ["", ""];
    // prefix for file names
    const baseURL = "./_____";

    // iterate over file names and upload one at a time
    for (let i = 0; i < filesToUpload.length; i++) {
        const data = fs.readFileSync(baseURL + filesToUpload[i]);

        // create a Bundlr Transaction
        const tx = bundlr.createTransaction(data);

        // want to know how much you'll need for an upload? simply:
        // get the number of bytes you want to upload
        const size = tx.size;

        // query the bundlr node to see the price for that amount
        const cost = await bundlr.getPrice(size);
        console.log("cost= ", cost.toString());
        console.log("decimal cost= ", bundlr.utils.unitConverter(cost).toString());

        // do we need more money?
        // Lazy fund the wallet, meaning each time we check how much to add and then add it
        // An alternative would be to first estimate cost (see price-check.js) and then
        // fund the account for all files at once.
        // According to the docs, "this approach only works effectively for currencies with
        // fast confirmation times like SOL, MATIC etc."
        // Depending on your funding currency, you may have to fund in advance
        if (balance.isGreaterThan(cost)) {
            console.log("funding wallet");
            // NOTE: The online example at https://docs.bundlr.network/docs/client/examples/funding-your-account
            // does not use Math.ceil(), and results in an error saying "must use an integer for funding amount".
            // I added Math.ceil() to round up, making sure we always have plenty of dev funds.
            await bundlr.fund(Math.ceil(balance.minus(cost).multipliedBy(1.1)));
        }

        // sign the transaction
        await tx.sign();

        // get the transaction's ID:
        const id = tx.id;

        // upload the transaction
        const result = await tx.upload();

        // and voila!
        console.log(`${filesToUpload[i]} available at https://arweave.net/${id}`);
    }
}
main();

We start out connecting to the Bundlr Devnet while providing a URL to the Bundlr network, a currency to pay in, a wallet private key and devnet RPC. Since I'm paying in testnet MATIC, I provide a URL to Mumbai testnet (get a free one from Alchemy) const bundlr = new Bundlr.default("https://devnet.bundlr.network", "matic", process.env.PRIVATE_KEY, { providerUrl: process.env.MUMBAI_RPC, });

I then take a list of file names and a directory prefix, iterate over those files and upload them one at a time. The code is commented really well, so you probably don't need me to dig in too deep ... but just to be extra clear, here's the four main functions I'm working with.

I start out by creating a transaction using data read from the file system, get the price to store it, fund my account (if needed), sign the transaction and upload it.

const tx = bundlr.createTransaction(data);
const cost = await bundlr.getPrice(size);
await bundlr.fund(Math.ceil(balance.minus(cost).multipliedBy(1.1)));
await tx.sign();
const result = await tx.upload();

The script ends by printing the file name and new perma-url to the console.

OK, back to my project. I took all my MP3 files and used the script to upload them to Bundlr. Then I modified my dApp to use the new https://arweave.com URL and recorded this video. All the audio you hear is from files streamed directly from Bundlr / Arweave.

When I conceived of this project, I wanted the NFTs to have a life outside of my dApp. I wanted owners of the NFT to feel like they own a unique example of sound design and can do whatever they want with it. People can use the sounds in games or in other ways I would never have thought of. By moving the MP3 files to Bundlr, I'm able to guarantee the audio files will always be there, regardless of what happens to the dApp I built.

What's really important here is the files load quickly. To create evolving generative music, my dApp is constantly loading and unloading new instruments and everything just flowed seamlessly.

Ok, ok ... so the big question ... how much does it cost? Very very little.

To figure out pricing, I grabbed 8 files from my computer ranging in size from 256 bytes to 1.345728 megabytes. I used a slimmed down version of the code above, this time connecting to the main Bundlr network and specifying MATIC as a currency. As I didn't want to spend any money, my script only loads the file and checks the price to upload it. I don't fund the wallet or actually upload anything.

// import the client
import Bundlr from "@bundlr-network/client";
import fs from "fs";
import dotenv from "dotenv";
dotenv.config();

/**
 * @notice Script to check the price of variously sized files when uploaded
 * to the Bundlr permaweb.
 */
async function main() {
    // initialise a bundlr client
    // NOTE: The online example (https://docs.bundlr.network/docs/client/examples/full-example)
    // uses the following: bundlr = new Bundlr(...) which throws an error.
    // To run properly from node, you need to change to the version I have below.
    // This is documented here https://github.com/Bundlr-Network/js-sdk/issues/50
    const bundlr = new Bundlr.default("http://node1.bundlr.network", "matic", process.env.PRIVATE_KEY);

    // Hardcoded the MATIC price here to make things easy. Make sure to manually update
    // when running to get an accurate USD cost.
    const MATIC_PRICE = 0.885502;

    // files to check price of
    const filesToUpload = [
        "Greeter.sol",
        "README.md",
        "colors.png",
        "mallet-mellow-A3.mp3",
        "Space.png",
        "CryptoPunks.png",
        "CoffeeExchange.jpg",
        "CoffeeExchange.png",
    ];
    const baseURL = "./price-check/";
    let prices = [];
    prices.push(["File", "Size (bytes)", "Cost (Matic)", "Cost (USD)"]);

    for (let i = 0; i < filesToUpload.length; i++) {
        const data = fs.readFileSync(baseURL + filesToUpload[i]);

        // create a Bundlr Transaction
        const tx = bundlr.createTransaction(data);

        // want to know how much you'll need for an upload? simply:
        // get the number of bytes you want to upload
        const size = tx.size;

        // query the bundlr node to see the price for that amount
        let cost = await bundlr.getPrice(size);
        cost = bundlr.utils.unitConverter(cost).toString();
        let usdCost = cost * MATIC_PRICE;

        prices.push([filesToUpload[i], size, cost, usdCost]);
    }

    console.table(prices);
}
main();

When run, the results are as follows.

I was curious if different payment options would result in different prices, however when I ran the script using ETH as a currency, I got the same results.

Ok, that's all for today. The full code is on GitHub. I included the 8 files I used to check prices to make it easy for you to run the script in the future. I don't know how Bundlr is pricing things at all. Don't know if they have a base price in USD that varies depending on storage costs or if the price is fixed in AR and USD prices fluctuate? No idea at all here, but with these files in GitHub it's going to be easy to rerun the script and compare against future prices.