Creating an AI photo generator and editing app with React

About a year ago, ChatGPT wowed the world with its ability to generate text content on demand. But generative AI goes beyond text — it can also edit and create images using text prompts.

In this guide, I will show you how to build a web application using cloud-based AI with features like generating photorealistic images, removing backgrounds and objects from images, and facial restoration.

In the process of building this application, you will learn how to leverage cloud-based AI models to create innovative web solutions. With these skills, you are set to build real-world, AI-driven applications and contribute to the evolving field of AI.

Why cloud-based AI?

Replicate lets you run machine learning models with a cloud API without having to understand the intricacies of machine learning or managing your own infrastructure. Their tools start at $0.00010/sec, with a free trial plan.

Because Replicate is cloud-based and has a huge community, you can access over a thousand AI models that are dedicated to performing extraordinary things. Additionally, you don’t need to learn Python or R to build a model from scratch. Simply use your favorite language (JavaScript or TypeScript) to interact with these AI models.

The application we will build in this tutorial will make use of the following AI models:

• stable-diffusion: A latent text-to-image diffusion model capable of generating photorealistic images given any text input
• object-removal: A model that removes specified objects from an image
• rembg: A model that removes the background from images
• gfpgan: A practical face restoration model for old photos

If you’re ready to unlock the potential of AI in image generation and editing, let’s get started!

Prerequisites

To follow this article, I recommend you have foundational experience with TypeScript, React, and HTML/CSS.

Setting up our project

To begin, we will set up our project using Vite. To use Vite, first install Node on your machine if it’s not already, then set up a React project with this command:

npm create vite@latest

Vite will prompt you to provide extra information like the project name, framework, and programming language to complete the operation. If successful, you should have something similar to the terminal output shown below:

Now, cd into the project root directory and run npm install. Afterwards, run npm run dev to serve your React app on a local host.

Installing external dependencies

Next, we will install external dependencies for Ant Design and Axios.

Ant Design (antd) is a React component library for building beautiful and modern user interfaces. It comes with a collection of prebuilt, enterprise-level UI components. To install Ant Design, use the command below:

npm install antd --save

Axios is a powerful HTTP client library for making network requests from your browser.

npm install axios

Cleaning up the App component

By default, the App component has some code we don’t need. For now, replace it with the code block below:

function App() {

  return <div>Hello world</div>
}

export default App

Additionally, delete the App.css file from your source folder.

Updating default styling in index.css

In the index.css file, update the styling with the following:

body {
    margin: 0;
    padding: 0;
}

.layout {
  height: 100vh;
}

.site-layout .site-layout-background {
  background: #fff;
}

We are replacing all the CSS styles that came with the application with ours so that we have full control over the application’s appearance. If you run the application again, you should see Hello world positioned at the top left of the web page.

Setting up the application network layer

Generating a Replicate API key

An API key is needed to access resources on Replicate’s server. To create one, sign up here.

Defining the cloud AI models

The cloud AI models are identified by their unique IDs, which are constant. So, we will group them using an enum:

export enum ReplicateAIModels {
    TextToImage = "2b017d9b67edd2ee1401238df49d75da53c523f36e363881e057f5dc3ed3c5b2",
    BgRemover = "fb8af171cfa1616ddcf1242c093f9c46bcada5ad4cf6f2fbe8b81b330ec5c003",
    ObjRemover = "153b0087c2576ad30d8cbddb35275b387d1a6bf986bda5499948f843f6460faf",
    FaceRestorer = "297a243ce8643961d52f745f9b6c8c1bd96850a51c92be5f43628a0d3e08321a"
}

Generating an ImgBB API key

We need a temporary image service to host images that will be fed into the cloud AI models. Create an account on ImgBB to get a key that we will use to communicate with their API.

Defining constant keys

Create a constant.ts file with the following constants:

export const API_KEY = import.meta.env.VITE_REPLICATE_API_KEY
export const IMG_BB_API_KEY = import.meta.env.VITE_IMG_BB_API_KEY
export const BASE_URL = 'https://api.replicate.com/v1/'
export const PROXY_SERVER_URL = "https://proxy-server-407001.ue.r.appspot.com/"
export const IMGBB_URL = "https://api.imgbb.com/1/upload"

Because this project is hosted on GitHub, I am using the import.meta.env object from Vite to read environment variables. For testing purposes, you can include your keys.

You might be wondering why we need both BASE_URL and PROXY_URL. In order to access resources on Replicate’s server from a web browser, we need to bypass the browser’s CORS policy with a proxy server, hence the PROXY_SERVER_URL. Feel free to test without the proxy URL to see the error for yourself.

Implementing image upload

Within your source folder, create a ImageHostingService.ts file with the following code:

export const uploadImageToImgBB = async (image: RcFile) => {
    try {
        const formData = new FormData();
        formData.append('image', image);

        const response = await axios.post(IMGBB_URL, formData, {
            params: {
                expiration: 600,
                key: IMG_BB_API_KEY,
            },
            headers: {
                'Content-Type': 'multipart/form-data'
            },
        });

        return response.data\["data"\]["image"]["url"]
    } catch (error) {
        if (axios.isAxiosError(error)) {
            console.error('Request failed with status code:', error.response?.status);
            console.error('Response data:', error.response?.data);
        }
        throw "Error uploading image"
    }
}

Within the uploadImageToImgBB function block, we use Axios to make a multi-part request for uploading an image to the ImgBB server. Once the upload is successful, we extract the image URL from the response body. In the next section, we will see where this function is called.

Implementing the cloud AI API client

import axios, {AxiosInstance} from 'axios';
import {ReplicateAIModels} from "./CloudAiModels.ts";
import {RcFile} from "antd/lib/upload";
import {uploadImageToImgBB} from "./ImageHostingService.ts";
import {delay} from "../common/Utils.ts";
import {PredictionResponse} from "./Response.ts";
import {API_KEY, BASE_URL, PROXY_SERVER_URL} from "./Constants.ts";

class ApiClient {
    private axiosInstance: AxiosInstance

    constructor(baseURL: string) {
        this.axiosInstance = axios.create({
            baseURL,
            headers: {
                'Content-Type': 'application/json',
                'Accept': "application/json",
                "Authorization": `Token ${API_KEY}`
            },
        })

        this.setUpPredictionStatusPolling()
    }

    private setUpPredictionStatusPolling() {
        this.axiosInstance.interceptors.response.use(async (response) => {
                console.log("Interceptor", response)

                const predictionId = response.data["id"]
                const predictionUrl = `predictions/${predictionId}`;

                let pollingStatus = response.data["status"]

                while (pollingStatus === "starting" || pollingStatus === "processing") {
                    await delay(3000)
                    response = await this.axiosInstance.get(predictionUrl)
                    pollingStatus = response.data["status"]
                    console.log("Interceptor polling status: ", pollingStatus)
                }

                return response;
            }, (error) => Promise.reject(error)
        )
    }

    public async generateImage(userPrompt: string): Promise<PredictionResponse> {
        const data = {
            "version": ReplicateAIModels.TextToImage,
            "input": {
                "prompt": userPrompt
            }
        };

        return await this.createPrediction(data)
    }

    public async removeBackgroundFromImage(image: RcFile) {

        const url = await uploadImageToImgBB(image)

        const data = {
            "version": ReplicateAIModels.BgRemover,
            "input": {
                "image": url
            }
        };

        return await this.createPrediction(data)

    }

    public async removeObjectFromImage(image: RcFile, objectToRemove: string) {

        const url = await uploadImageToImgBB(image)

        const data = {
            "version": ReplicateAIModels.ObjRemover,
            "input": {
                "image_path": url,
                'objects_to_remove': objectToRemove
            }
        };

        return await this.createPrediction(data)
    }

    public async restoreImage(image: RcFile) {

        const url = await uploadImageToImgBB(image)

        const data = {
            "version": ReplicateAIModels.FaceRestorer,
            "input": {
                "img": url,
                "scale": 1,
            }
        };

        return await this.createPrediction(data)
    }

    async createPrediction(data: object): Promise<PredictionResponse> {
         const response = await this.axiosInstance.post<PredictionResponse>("/predictions", data)
        return response.data
    }
}


const apiClient = new ApiClient(PROXY_SERVER_URL + BASE_URL)
export default apiClient

Pay attention to the createPrediction and setUpPredictionStatusPolling functions in the code block above, as they encapsulate the core business logic of the application.

The function createPrediction submits an image processing request to Replicate’s server via the prediction endpoint. Because image processing takes time, the server returns a response immediately with a Job ID (Prediction ID), which we will use to query the Replicate server for the task status at a later time.

Within setUpPredictionStatusPolling, we implement a polling logic that is common to all the image processing tasks we submit to Replicate’s server. Inside the response interceptor for the prediction endpoint, we query the server for the prediction status every three seconds.

If the status isn’t in the starting or processing phases, we return the response from the interceptor. By adopting this approach, we make sure there’s a single point of logic handling both the polling of prediction status and retrieving the task result.

Outside of the scope of the ApiClient class, we created an instance of the same class and made it visible to other files within our source code. It is a single instance of the API client, which will be shared by all the components that will be created in later sections.

Setting up the UI layer

Having set up our core business logic, we are now going to create the user interfaces that will depend on them. As you already know, we will rely on React and UI components from Ant Design.

Implementing the Imagine component

Within the Imagine component, we are going to implement the user interface for generative photo creation. With only text prompts, we’ll be able to generate realistic images. Hang tight, we’ll achieve that in a few lines of code:

import {useState} from "react";
import {Flex, Image} from "antd";
import Search from "antd/lib/input/Search";
import apiClient from "../api/ReplicateApiClient.ts";

const Imagine = () => {

    const [imgSrc, setImgSrc] = useState("")
    const [loading, setLoading] = useState(false)

    function generateImage(query: string) {
        setLoading(true)
        apiClient.generateImage(query)
            .then((data) => {
                const imageUrl = data.output[0];
                setImgSrc(imageUrl)
                setLoading(false)
            })
            .catch(() => {
                setLoading(false)
            })
    }

    return (
        <Flex vertical={true} gap={10}>
            <Search
                style={{width: "500px"}}
                placeholder="Describe an image to generate"
                enterButton="Submit"
                disabled={loading}
                loading={loading}
                onSearch={(value) => {
                    (value.trim().length > 5) && generateImage(value)
                }}
            />

            {(imgSrc !== '') && (
                <Image
                    height="50%"
                    width="50%"
                    src={imgSrc}
                    alt="Your Image Alt Text"
                />
            )}
        </Flex>
    )
}

export default Imagine;

The Imagine function is composed of a Search input component and an Image component. Whenever imgSrc changes state based on user text input, React updates the Image component to reflect the latest image generated:

Implementing the BackgroundRemove component

The BackgroundRemove component encapsulates the logic for removing backgrounds from images. With one click, you can effortlessly remove distracting backgrounds from your photos without the hassle of an eraser tool. See the code block below for the implementation:

import {useState} from 'react';
import {RcFile} from "antd/lib/upload";
import {isString} from "antd/es/button";
import {UploadOutlined} from "@ant-design/icons";
import {Button, Flex, Image, Upload} from "antd";
import apiClient from "../api/ReplicateApiClient.ts";

const BackgroundRemover = () => {

    const [image, setImage] = useState<RcFile>()
    const [imageAsUrl, setImageAsUrl] = useState("")
    const [editedImgSrc, setEditedImgSrc] = useState("")
    const [loading, setLoading] = useState(false)

    function handleChange(file: RcFile) {
        setImage(file)
        setImageAsUrl(URL.createObjectURL(file));
    }

    function removeImageBackground() {
        setLoading(true)
        apiClient.removeBackgroundFromImage(image as RcFile)
            .then((data) => {
                const imageUrl = data.output;
                if (isString(imageUrl)) setEditedImgSrc(imageUrl)
                setLoading(false)
            })
            .catch(() => {
                setLoading(false)
            });
    }

    return (

        <Flex vertical={true} gap={10}>

            <Flex vertical={false} gap={10}>

                <Upload
                    showUploadList={false}
                    disabled={loading}
                    beforeUpload={handleChange}>
                    <Button
                        disabled={loading}
                        icon={<UploadOutlined/>}
                        size={"middle"}>Import photo</Button>
                </Upload>

                {(imageAsUrl !== "") && <Button
                    type="primary"
                    disabled={loading}
                    loading={loading}
                    onClick={removeImageBackground}
                    size={"middle"}>
                    Remove Background
                </Button>}


            </Flex>


            <Flex vertical={false} gap={40}>

                {(imageAsUrl !== "") && (
                    <Image
                        height="40%"
                        width="40%"
                        src={imageAsUrl}
                        alt="Your Image Alt Text"
                    />
                )}

                {(editedImgSrc !== '') && (
                    <Image
                        height="40%"
                        width="40%"
                        src={editedImgSrc}
                        alt="Your Image Alt Text"
                    />
                )}
            </Flex>

        </Flex>

    )
}

export default BackgroundRemover

Ant Design’s Upload component responds to clicks by opening the file window for image selection. Within this flow, the selected file is passed to handleChange(file) in order to set the image for preview.

Whenever removeImageBackground() is triggered from an onClick() button event, we invoke removeBackgroundFromImage(image) on the apiClient instance. The response from the method returns a promise with which we register a callback that will be triggered when the promise is fulfilled. Within this success callback, we extract imageUrl and update the state of editedImgSrc.

As usual, React updates the DOM to show the selected photo with its background removed:

 

Implementing the ObjectRemover component

Using the generative removal tool from Replicate, you can effortlessly remove unwanted objects from your image with only simple prompts. An example use case might be to remove a car from an otherwise perfect property photo.

Within the ObjectRemover component, we will build the interface and business logic that allows for object removal:

import {Button, Flex, Image, Upload} from "antd"
import {UploadOutlined} from "@ant-design/icons"
import {useState} from "react"
import {RcFile} from "antd/lib/upload"
import apiClient from "../api/ReplicateApiClient.ts"
import Search from "antd/lib/input/Search"
import {isString} from "antd/es/button"

const ObjectRemover = () => {

    const [image, setImage] = useState<RcFile>()
    const [imageAsUrl, setImageAsUrl] = useState("")
    const [editedImgSrc, setEditedImgSrc] = useState("")
    const [loading, setLoading] = useState(false)

    function handleChange(file: RcFile) {
        setImage(file)
        setImageAsUrl(URL.createObjectURL(file))
    }

    function removeObjectFromPhoto(objectDescription: string) {
        setLoading(true)
        apiClient.removeObjectFromImage(image as RcFile, objectDescription)
            .then((data) => {
                if (isString(data.output)) {
                    setEditedImgSrc(data.output)
                }
                console.log("response", data)
                setLoading(false)

            }).catch((error) => {
            console.error("error", error)
            setLoading(false)

        })
    }

    return (

        <Flex vertical={true} gap={10}>

            <Flex vertical={false} gap={10}>


                <Upload
                    showUploadList={false}
                    disabled={loading}
                    beforeUpload={handleChange}>
                    <Button
                        disabled={loading}
                        icon={<UploadOutlined/>}
                        size={"middle"}>Import photo</Button>
                </Upload>

                {(image !== null) && <Search
                    style={{width: "20vw"}}
                    placeholder="Objects(s) to remove"
                    enterButton="Submit"
                    disabled={loading}
                    loading={loading}
                    onSearch={(value) => {
                        (value.trim().length > 0) && removeObjectFromPhoto(value)
                    }}
                />

                }

            </Flex>

            <Flex vertical={false} gap={40}>

                {(imageAsUrl !== "") && (
                    <Image
                        height="40%"
                        width="40%"
                        src={imageAsUrl}
                        alt="Your Image Alt Text"
                    />
                )}

                {(editedImgSrc !== '') && (
                    <Image
                        height="40%"
                        width="40%"
                        src={editedImgSrc}
                        alt="Your Image Alt Text"
                    />
                )}
            </Flex>

        </Flex>
    )
}

export default ObjectRemover

The ObjectRemover component is composed of an Upload component for selecting an image and a Search component for capturing user prompts. In response to the imported image and user prompt, apiClient.removeObjectFromImage(image, objectDescription) is invoked with the appropriate arguments for requesting object removal.

The result from the operation will be the input image minus the objects from the user prompt:

Implementing the Restaurer component

Restoring details on blurry photos can be a daunting and time-consuming task, but with cloud AI models, we can perform photo restoration in a fraction of the time. The Restaurer component contains logic to enable photo restoration in one click:

import {useState} from 'react';
import {Button, Flex, Image, Upload} from "antd";
import {UploadOutlined} from "@ant-design/icons";
import {RcFile} from "antd/lib/upload";
import apiClient from "../api/ReplicateApiClient.ts";
import {isString} from "antd/es/button";

const Restaurer = () => {

    const [image, setImage] = useState<RcFile>()
    const [imageAsUrl, setImageAsUrl] = useState("")
    const [editedImgSrc, setEditedImgSrc] = useState("")
    const [loading, setLoading] = useState(false)

    function handleChange(file: RcFile) {
        setImage(file)
        setImageAsUrl(URL.createObjectURL(file));
    }

    function restoreImage() {
        setLoading(true)
        apiClient.restoreImage(image as RcFile)
            .then((data) => {
                const imageUrl = data.output;
                if (isString(imageUrl)) setEditedImgSrc(imageUrl)
                setLoading(false)
            })
            .catch(() => {
                setLoading(false)
            })
    }

    return (

        <Flex vertical={true} gap={10}>

            <Flex vertical={false} gap={10}>

                <Upload
                    showUploadList={false}
                    disabled={loading}
                    beforeUpload={handleChange}>
                    <Button
                        disabled={loading}
                        icon={<UploadOutlined/>}
                        size={"middle"}>Import photo</Button>
                </Upload>

                {(imageAsUrl !== "") && <Button
                    type="primary"
                    disabled={loading}
                    loading={loading}
                    onClick={restoreImage}
                    size={"middle"}>
                    Restore Photo
                </Button>}

            </Flex>

            <Flex vertical={false} gap={40}>

                {(imageAsUrl !== "") && (
                    <Image
                        height="40%"
                        width="40%"
                        src={imageAsUrl}
                        alt="Original photo"
                    />
                )}

                {(editedImgSrc !== '') && (
                    <Image
                        height="40%"
                        width="40%"
                        src={editedImgSrc}
                        alt="Edited Photo"
                    />
                )}
            </Flex>

        </Flex>
    )
}

export default Restaurer

The code block is similar to that of the RemoveBackground component, the only difference being the method called on the apiClient instance. Successful execution of restoreImage on apiClient returns a new image with improvements to the originally imported photo. See the image below for a comparison:

Setting up the App component

The final piece of work involves a sidebar navigation menu within the App component. This setup enables a seamless switching between top-level components:

import {BlockOutlined, BoldOutlined, FileImageOutlined, RotateLeftOutlined} from '@ant-design/icons'
import {Layout, Menu, MenuProps} from 'antd'
import {useState} from "react"
import Imagine from "./components/Imagine.tsx"
import ObjectRemover from "./components/ObjectRemover.tsx"
import BackgroundRemover from "./components/BackgroundRemover.tsx"
import Restaurer from "./components/Restaurer.tsx"

const {Sider, Content} = Layout

function App() {

    const menuItems = [
        {
            key: '1',
            icon: <FileImageOutlined/>,
            label: 'Imagine'
        },
        {
            key: '2',
            icon: <BlockOutlined/>,
            label: 'Obj Remover',
        },
        {
            key: '3',
            icon: <BoldOutlined/>,
            label: 'Bg Remover',
        },
        {
            key: '4',
            icon: <RotateLeftOutlined/>,
            label: 'Restaurer',
        },
    ]

    const sideBarComponents = [
        <Imagine/>,
        <ObjectRemover/>,
        <BackgroundRemover/>,
        <Restaurer/>
    ]

    const [itemId, setItemId] = useState("1")

    const onClickHandler: MenuProps['onClick'] = (e) => {
        setItemId(e.key)
    }

    return (
        <Layout className="layout">
            <Sider trigger={null} collapsible width={200}>
                <div className="logo">Cloud-IMG</div>
                <Menu
                    theme={"dark"}
                    onClick={onClickHandler}
                    mode="inline"
                    defaultSelectedKeys={['1']}
                    items={menuItems}
                />
            </Sider>
            <Layout className="site-layout">
                <Content
                    className="site-layout-background"
                    style={{
                        margin: '24px 16px',
                        padding: 24,
                    }}
                >
                    {sideBarComponents[parseInt(itemId) - 1]}

                </Content>
            </Layout>
        </Layout>
    )
}

export default App

The layout component defines a scaffold that hosts the Sider (the sidebar) and Content components. This simplifies web page setup for complex user interfaces.

On the sidebar, whenever a menu item is clicked, the state of the itemId is set to the item’s key value. This change triggers React’s reconciliation process and updates the content component visually.

With the App component fully set up, the web app should look like the image below:

Demoing our application

Here’s a video recording of the final application, demonstrating the different cloud AI use cases we covered in the article:

Cloud-AI Image Manipulation and Generation, react.js demo

This react application interacts with AI models on replicate.

 

Feel free to explore the live demo of the application.

Conclusion

AI is here to stay, and cloud-based AI models have unlocked boundless possibilities for software applications. Through this article, we delved into the world of AI models, equipping you with the skills to build a remarkable photo editing application using React.

Beyond this, I want to challenge you to explore other cloud-based AI models on Replicate and discover interesting project ideas you can build.