Nausheen Sujela Nausheen is a software-engineer-and-writer-hybrid who uses her cross-disciplinary experience to craft compelling copy and content about all things technology and marketing.

Introduction to Pipedream

3 min read 1118

Intro to Pipedream Feature Image

Pipedream is a free, low-code integration tool that allows developers to connect many different applications, data sources, and APIs in order to build useful, automated cross-platform workflows. It’s easy to use, and there’s no need to manage infrastructure or server resources.

Examples of workflows you can build with Pipedream include:

In this article, you’ll learn about key Pipedream features, what apps and APIs can be integrated (spoiler: there are many!), and how you can start building your own workflows today.

Notable capabilities of Pipedream

1. Build event streams using popular APIs

You can turn any API into an event stream, and any event stream into an API. Use Pipedream to collect your favorite services’ data — Tweets, RSS feeds, Stripe events, GitHub notifications, webhook events, Airtable records, the bitcoin blockchain, and more — and trigger new events based on the data received.

2. Run complex integrations with Node.js

Pipedream is easy enough to use as a no-code platform but powerful enough to enable developers to create custom workflows and solutions. Workflows are made up of Node.js steps that are triggered upon every event.

You can develop your own Node.js code or use the various built-in actions provided (e.g., sending a message to Slack or storing data in Snowflake) without writing any code. Pipedream allows you to schedule workflows or trigger them via a URL, email address, or SDK code.

3. Built-in authentication, state management, and observability

  • Authentication: OAuth and key-based authentication are both supported, with OAuth flow and token refresh fully handled by Pipedream.
  • State management: Pipedream enables you to maintain state between serverless executions with a single line of code.
  • Observability: Access workflow output, errors, timing, and return values to help you observe events and debug executions in real time. Pipedream provides visibility into previous versions of the code to enable easy troubleshooting.

4. Run SQL on the Pipedream data warehouse

Pipedream lets you run SQL on event data for free. You don’t have to create SQL tables or schema — you can just send it in, run your SQL commands, and even join tables across multiple workflows.

5. Deliver data quickly and easily

Pipedream allows you to asynchronously send data to HTTP endpoints, Snowflake, S3, and other sources. There’s no need to worry about batching, compression, or connection logic because Pipedream handles all of that while you just focus on the data.

Key components of Pipedream

1. Workflows

A workflow is essentially a series of steps that are executed sequentially. These steps can include retrieving data or sending data somewhere. You can add steps by writing more Node.js code or taking advantage of hundreds of pre-built actions like “search Twitter” or “send SMS message.”

By default, workflow code is entirely public and free for developers to copy and use. Your own workflow execution and resulting event data is, however, private.

We made a custom demo for .
No really. Click here to check it out.

2. Event sources

Event sources lets you find and collect data from sources like HTTP or RSS, as well as services like the bitcoin blockchain, Stripe, GitHub, Airtable, Reddit, and more. You can use these data streams to create new events and trigger workflows. Alternatively, you can consume event streams using Pipedream’s REST API.

3. Workflow steps

The building blocks used to create Pipedream workflows are referred to as steps. There are three types of steps: triggers, code, and actions.

  • Triggers: Triggers are the initial steps that kick off the execution of a workflow. Currently supported triggers include email, HTTP URLs, SDK endpoints, and a scheduler, among others. Pipedream plans to support triggers for SQL, AMQP, and more in the future.
  • Code: Code steps are optional but powerful for developers that want to build custom solutions. Code steps are essentially Node.js scripts (Pipedream currently supports Node.js v10) and can make use of most of npm’s 400,000 packages.
  • Actions: Actions are reusable code steps that connect and integrate your data, APIs, and applications. There are hundreds of built-in actions provided by Pipedream, and you can also create and publish your own.

4. List of Integrated Apps

Pipedream currently supports 250+ apps, so you can integrate across hundreds of your favorite services, including: Twitter, G Suite, GitHub, Slack, Asana, Discord, Gmail, Reddit, Ghost, Bitbucket, Netlify, and many, many more.

Tutorial: Create your first workflow

Setting up your first Pipedream workflow is simple and won’t take more than five minutes.

To get started, head over to Pipedream and sign in with your GitHub or Google account.

You’ll eventually land on the page where you’ll build your workflow. It looks something like this:

Pipedream Workflow Page Connected to GitHub

When you click the + sign at the bottom of the window, you’ll open up a list containing thousands of pre-built, reusable actions offered by Pipedream.

Pipedream pre-built options

These pre-built actions include but are not limited to:

  • Searching Twitter
  • Making a pull request on GitHub
  • Retrieving changes for an individual or shared Google Drive
  • Creating and deleting Google Drive files
  • Creating Zoom meetings
  • Opening and sending messages on Slack
  • Emailing invoices via Zoho Books
  • Creating and retrieving Airtable records

For our example, let’s add a step to search for GitHub repositories. Immediately after clicking the action, a cell is created on the workflow page and we’re prompted to connect to GitHub. Click on Connect GitHub.

Once you sign in with your GitHub account, enter your search term in the params box. Let’s suppose we want to find repositories for clock applications; we then type clock into the params box.

Pipedream GitHub Param Box

After you’re done, hit Send test event, and boom! You’ve just created your first workflow. Here’s what the result looks like:

Pipedream Workflow Test Event

In a similar fashion, you can add multiple steps across several applications. For example, perhaps you want to be notified every time there’s a new “clock” repository on GitHub. In that case, we could schedule the above action to run every few hours and select a pre-built email action to send ourselves an email reminder every time our workflow discovers a new clock repository.

Conclusion

Pipedream offers a clean, convenient way for developers to automate tasks. Whether you want to send yourself email reminders, connect apps to Google Calendar, or receive real-time notifications on your favorite platforms, Pipedream provides hundreds of app integrations, built-in applications, and powerful customization capabilities to help you create workflows that work for you.

There is also extensive documentation available, as well as a growing developer community to help out with any questions or issues you might run into.

Overall, Pipedream provides a powerful solution for developers looking to build cross-platform pipelines with minimal code and automate their day-to-day tasks and activities. To get started, check out Pipedream’s official website, as well as their developer docs here.

You come here a lot! We hope you enjoy the LogRocket blog. Could you fill out a survey about what you want us to write about?

    Which of these topics are you most interested in?
    ReactVueAngularNew frameworks
    Do you spend a lot of time reproducing errors in your apps?
    YesNo
    Which, if any, do you think would help you reproduce errors more effectively?
    A solution to see exactly what a user did to trigger an errorProactive monitoring which automatically surfaces issuesHaving a support team triage issues more efficiently
    Thanks! Interested to hear how LogRocket can improve your bug fixing processes? Leave your email:

    : Full visibility into your web apps

    LogRocket is a frontend application monitoring solution that lets you replay problems as if they happened in your own browser. Instead of guessing why errors happen, or asking users for screenshots and log dumps, LogRocket lets you replay the session to quickly understand what went wrong. It works perfectly with any app, regardless of framework, and has plugins to log additional context from Redux, Vuex, and @ngrx/store.

    In addition to logging Redux actions and state, LogRocket records console logs, JavaScript errors, stacktraces, network requests/responses with headers + bodies, browser metadata, and custom logs. It also instruments the DOM to record the HTML and CSS on the page, recreating pixel-perfect videos of even the most complex single-page apps.

    .
    Nausheen Sujela Nausheen is a software-engineer-and-writer-hybrid who uses her cross-disciplinary experience to craft compelling copy and content about all things technology and marketing.

    Leave a Reply