2022-05-05
2868
#aws#node#react
Antonello Zanini
105616
May 5, 2022 ⋅ 10 min read

Multipart uploads with S3 in Node.js and React

Antonello Zanini I'm a software engineer, but I prefer to call myself a technology bishop. Spreading knowledge through writing is my mission.

Recent posts:

Implementing Vector Search With Open Ai, Next Js, And Supabase

Implementing vector search with OpenAI, Next.js, and Supabase

Let’s build a Next.js app that implements vector search using Supabase and OpenAI to offer better search experiences for users.

Peter Ekene Eze
Feb 27, 2024 ⋅ 11 min read
Comparing React Native BLE libraries

Comparing React Native BLE libraries

Discover the most popular libraries for enabling Bluetooth Low Energy (BLE) in React Native apps, allowing them to interact with Bluetooth hardware on iOS and Android platforms.

Fimber Elemuwa
Feb 26, 2024 ⋅ 4 min read
Using CRDTs To Build Collaborative Rust Web Applications

Using CRDTs to build collaborative Rust web applications

CRDTs, or conflict-free replicated data types, is a concept that underlies applications facing the issue of data replication across a […]

Mario Zupan
Feb 23, 2024 ⋅ 15 min read
Guide to Using TensorFlow in Rust

Guide to using TensorFlow in Rust

We explore the fusion of TensorFlow and Rust, delving into how we can integrate these two technologies to build and train a neural network.

Rosario De Chiara
Feb 22, 2024 ⋅ 8 min read
View all posts

22 Replies to "Multipart uploads with S3 in Node.js and React"

  1. When I try to access the etag header on the upload response, I get “Refused to get unsafe header “ETag”. Seems the destination has cors configured and will not let me get the etag. Not sure how you got around this problem?

  2. Hi, I am the author of the article!
    If you get the error when trying to upload a portion of the file with a pre-signed URL, it means that it must be due to a wrong configuration on S3.

  3. i cloned your repo to run locally. It doesn’t seem to work for me when I try uploading a file > 5MB; it gets stuck in the middle(~35%, depending on the file size). Am I missing anything?

    1. Hey! It’s the author of the article here.
      Could you please expand on what error you get?
      Also, make sure your S3-like provider support multiple upload and your Internet connection doesn’t fail.

  4. Thank you for your response. I was able to fix the issue by adding “ExposeHeaders”: [ “ETag”] in the bucket CORS policy. Adding it here for someone who might come across the same problem.

  5. What possible solution could be to make an API call from useEffect after the file is uploaded 100% and the finalizeMultipartUpload API is executed successfully? My API require file to be available in the bucket, and i don’t want to call the API inside the class. Any input would be appreciated.

    1. You could extend Uploader to accept a callback in videoUploaderOptions. This function will be called after
      await api.request({
      url: “/uploads/finalizeMultipartUpload”,
      method: “POST”,
      data: videoFinalizationMultiPartInput,
      })

  6. Nice article! Here presigned URLs were created by the service so we offloaded the server from upload work hence making it scalable solution. What do you think about the download part (multiple files as zip), should that also be done via presigned URLs?

    1. Thanks! I’m the author here!

      When it comes to downloading, presigned URLs are particularly useful to share files to users that don’t have access to the bucket. But, you could also use them to download a very large file, one chunk at a time.

  7. Hey I am getting this error. Proxy error: Could not proxy request /uploads/initializeMultipartUpload from localhost:3000 to http://localhost:3001.
    I have below config.
    frontend/uti/upload.js
    const api = axios.create({
    baseURL: “http://localhost:3000”,
    })
    backend/controller/upload.js
    const s3Endpoint = new AWS.Endpoint(“https://”)
    package.json
    “proxy”: “http://localhost:3001/”

      1. Hi Yes I started both npm run backend:start and npm run frontend:start.
        At browser its giving error as “XML Parsing Error: syntax error
        Location: http://localhost:3000/uploads/initializeMultipartUpload
        Line Number 1, Column 1:”

        At backend terminal its giving error: [nodemon] app crashed – waiting for file changes before starting…
        errno: -3008,
        code: ‘NetworkingError’,
        syscall: ‘getaddrinfo’,

        At frontend terminal its giving error as: Proxy error: Could not proxy request /uploads/initializeMultipartUpload from localhost:3000 to http://127.0.0.1:3001/.
        See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (ECONNRESET).

    1. Right now, it would faild, but you can easily change that by updating the login below:

      From
      this.sendChunk(chunk, part, sendChunkStarted)
      .then(() => {
      this.sendNext()
      })
      .catch((error) => {
      this.parts.push(part)

      this.complete(error)
      })

      To
      this.sendChunk(chunk, part, sendChunkStarted)
      .then(() => {
      this.sendNext()
      })
      .catch((error) => {
      this.parts.push(part)
      })

  8. cess to XMLHttpRequest at ” from origin ‘http://localhost:3000’ has been blocked by CORS policy: Response to preflight request doesn’t pass access control check: No ‘Access-Control-Allow-Origin’ header is present on the requested resource.

  9. Great article. I tested it and I’m getting a BAD REQUEST on the OPTIONS which is the CORS preflight done by the browser on the S3 bucket.
    As I debug it can you let me know if the OPTIONS request is supposed to be sent or maybe is some kind of configuration on my side which triggers it?

Leave a Reply