2022-05-05
2868
#aws#node#react
Antonello Zanini
105616
May 5, 2022 ⋅ 10 min read

Multipart uploads with S3 in Node.js and React

Antonello Zanini I'm a software engineer, but I prefer to call myself a technology bishop. Spreading knowledge through writing is my mission.

Recent posts:

Lots of multi-colored blue and purplish rectangles.

Animating dialog and popover elements with CSS @starting-style

Native dialog and popover elements have their own well-defined roles in modern-day frontend web development. Dialog elements are known to […]

Rahul Chhodde
Jul 24, 2024 ⋅ 10 min read
Using Llama Index To Add Personal Data To Large Language Models

Using LlamaIndex to add personal data to LLMs

LlamaIndex provides tools for ingesting, processing, and implementing complex query workflows that combine data access with LLM prompting.

Ukeje Goodness
Jul 23, 2024 ⋅ 5 min read
JavaScript logo on top of violet background

Exploring essential DOM methods for frontend development

Learn four groups of DOM methods and their uses to create responsive and dynamic webpages. A helpful DOM reference table is also included.

Chimezie Innocent
Jul 23, 2024 ⋅ 12 min read
Playwright Adoption Guide: Overview, Examples, And Alternatives

Playwright adoption guide: Overview, examples, and alternatives

Playwright has become one of the leading technologies for writing end-to-end tests for modern web apps thanks to its rich and intuitive API.

Antonello Zanini
Jul 22, 2024 ⋅ 15 min read
View all posts

22 Replies to "Multipart uploads with S3 in Node.js and React"

  1. When I try to access the etag header on the upload response, I get “Refused to get unsafe header “ETag”. Seems the destination has cors configured and will not let me get the etag. Not sure how you got around this problem?

  2. Hi, I am the author of the article!
    If you get the error when trying to upload a portion of the file with a pre-signed URL, it means that it must be due to a wrong configuration on S3.

  3. i cloned your repo to run locally. It doesn’t seem to work for me when I try uploading a file > 5MB; it gets stuck in the middle(~35%, depending on the file size). Am I missing anything?

    1. Hey! It’s the author of the article here.
      Could you please expand on what error you get?
      Also, make sure your S3-like provider support multiple upload and your Internet connection doesn’t fail.

  4. Thank you for your response. I was able to fix the issue by adding “ExposeHeaders”: [ “ETag”] in the bucket CORS policy. Adding it here for someone who might come across the same problem.

  5. What possible solution could be to make an API call from useEffect after the file is uploaded 100% and the finalizeMultipartUpload API is executed successfully? My API require file to be available in the bucket, and i don’t want to call the API inside the class. Any input would be appreciated.

    1. You could extend Uploader to accept a callback in videoUploaderOptions. This function will be called after
      await api.request({
      url: “/uploads/finalizeMultipartUpload”,
      method: “POST”,
      data: videoFinalizationMultiPartInput,
      })

  6. Nice article! Here presigned URLs were created by the service so we offloaded the server from upload work hence making it scalable solution. What do you think about the download part (multiple files as zip), should that also be done via presigned URLs?

    1. Thanks! I’m the author here!

      When it comes to downloading, presigned URLs are particularly useful to share files to users that don’t have access to the bucket. But, you could also use them to download a very large file, one chunk at a time.

  7. Hey I am getting this error. Proxy error: Could not proxy request /uploads/initializeMultipartUpload from localhost:3000 to http://localhost:3001.
    I have below config.
    frontend/uti/upload.js
    const api = axios.create({
    baseURL: “http://localhost:3000”,
    })
    backend/controller/upload.js
    const s3Endpoint = new AWS.Endpoint(“https://”)
    package.json
    “proxy”: “http://localhost:3001/”

      1. Hi Yes I started both npm run backend:start and npm run frontend:start.
        At browser its giving error as “XML Parsing Error: syntax error
        Location: http://localhost:3000/uploads/initializeMultipartUpload
        Line Number 1, Column 1:”

        At backend terminal its giving error: [nodemon] app crashed – waiting for file changes before starting…
        errno: -3008,
        code: ‘NetworkingError’,
        syscall: ‘getaddrinfo’,

        At frontend terminal its giving error as: Proxy error: Could not proxy request /uploads/initializeMultipartUpload from localhost:3000 to http://127.0.0.1:3001/.
        See https://nodejs.org/api/errors.html#errors_common_system_errors for more information (ECONNRESET).

    1. Right now, it would faild, but you can easily change that by updating the login below:

      From
      this.sendChunk(chunk, part, sendChunkStarted)
      .then(() => {
      this.sendNext()
      })
      .catch((error) => {
      this.parts.push(part)

      this.complete(error)
      })

      To
      this.sendChunk(chunk, part, sendChunkStarted)
      .then(() => {
      this.sendNext()
      })
      .catch((error) => {
      this.parts.push(part)
      })

  8. cess to XMLHttpRequest at ” from origin ‘http://localhost:3000’ has been blocked by CORS policy: Response to preflight request doesn’t pass access control check: No ‘Access-Control-Allow-Origin’ header is present on the requested resource.

  9. Great article. I tested it and I’m getting a BAD REQUEST on the OPTIONS which is the CORS preflight done by the browser on the S3 bucket.
    As I debug it can you let me know if the OPTIONS request is supposed to be sent or maybe is some kind of configuration on my side which triggers it?

Leave a Reply