Ingesting data should be easy, but all too often, it can be anything but. Data can come in many different formats, from many different sources, and wrangling it all can be a time-consuming and frustrating process. At Algolia, we believe that connecting data should be simple, and we’re committed to providing our customers with the tools they need to make that a reality. That’s why we’re excited to announce our new no-code connector platform with new JSON and CSV connectors! 

With this new platform, you can quickly and easily connect to the data you need, without having to worry about complex configurations or coding skills. Simply tell us where your data is located, how you want to ingest it and where you want it to go, and we’ll handle the rest. Whether you’re a data analyst, a business user, or a developer, our new no-code data connector platform can help simplify getting data into Algolia without any headaches or hassles.

Built for popular data types

Since JSON and CSV files are some of the most ubiquitous formats for data out there, we wanted some of our first few connectors to be focused on these file types. Whether you’re working with homegrown business processes, exports from your content management system, or just plain hosted files, our connectors can help you quickly and easily access the data you need. Each connector comes with an easy interface to set up as well as a matching API that allows you to fetch data from JSON and CSVs either on-demand or on a schedule. This means that you can spend less time struggling to connect to your data, and more time focusing on building amazing search and discovery experiences.

Ready to get started with our new no-code JSON and CSV connectors? It’s easy! In just a few simple steps, you can bring the data you need into your Algolia index. Follow along with this quick guide to setting up our JSON connector with a sample file of movie data: 

https://raw.githubusercontent.com/algolia/datasets/master/movies/records.json.

All you will need for this guide is that URL and an Algolia account. Your account must have permission to create and manage API keys within the app you are using. Before you begin, make sure you have an API key that you can use with the connector. The API key must have the addObject, deleteIndex, and editSettings permissions and will be used to move data into your indices.

To set up any connector, whether it’s JSON, CSV or otherwise, there are three parts to configure:

1. The data source: Configure the connector by indicating where your data is located, if there is any authentication needed, and how to identify a unique record.
2. The destination: Let us know which index you want to send this data to.
3. The task: How and when do you want to fetch the data? On-demand or on a schedule?

Let’s walk through each part step-by-step.

Configure your data source

First we need to create a new data source based on the movie data URL.

1. Visit the Connectors page in the Algolia dashboard to find our no-code connectors along with a variety of ways to connect your data.
2. Click the Connect button for the JSON connector. Then select Get started.
3. Let’s create a new source by first selecting the appropriate authentication type. Since our JSON file is fully open to the public we’ll select None as our authentication type. The JSON and CSV connectors currently support publicly available files and those protected with a username and password.
4. Next enter the URL listed above as our source of data. Our connectors support both HTTP and FTP-hosted files.
5. Now we need to configure the property that uniquely identifies a record in our data. This will often be some kind of ID, SKU, or similar unique field. In the sample movie data set, we already have a property called objectID that acts as the unique identifier so we will enter “objectID”. Whatever property you set as your unique identifier will be used as the objectID in your index.
6. Enter a name to identify this data source. This name should be easily recognizable and human-readable so you can reuse it in other areas of the connector platform.
7. Click Create source to create a new data source and move to the next step.

Configure your destination

Next we need to create a new destination that points at your index.

1. When sending data to your index we will use one of your own API keys to authenticate. Select a specific API key to use with this connector by first clicking on the “Search or create an authentication” input and then selecting “Create a new Algolia authentication”. Here you can choose which existing API key to use with the connector. The API key must have the following permissions: addObject, deleteIndex, and editSettings. If you do not see an eligible key listed, you will need to first make a new one on the API key settings page in the Algolia dashboard.
   
   
2. Once you’ve created an authentication, select an existing index or enter a specific name to create a new one.
3. Enter a name to identify this destination. Again, make sure it’s something easily identifiable so you can reuse it in other areas of the connector platform.
4. Click Create destination to create a new destination and move to the final step.
   
   

Configure your task

A task is the way to define how and when your data will be fetched and ingested.

1. For this guide we’ll set up the connector to fetch data on a daily basis. Select the Scheduled option to show the frequency input.
2. Choose an option from the drop-down menu’s list of popular schedules or enter a cron expression. To have Algolia fetch and update the data in our index daily we can simply select Every day.
3. Click Create task to create a new task.
   
   

And that’s it! You’ve now set up a connector and your data will be pushed into your index every day automatically! If you want to see it in action now, simply click the run button with the play icon to kick off your data task right now. If you created a new index in the setup process, you will now see it available with all the movie data!

We’re committed to providing you with a powerful and flexible connectivity platform so you can focus on building robust search and discovery experiences. We plan to continuously expand our no-code connector platform to include even more data sources and more data processing capabilities. Whether you’re working with CMS data, an ecommerce platform, or databases, we want to make it easy for you to connect to the data you need and be confident in your data ingestion pipeline. Visit the new Connectors page in your Algolia dashboard to see our new no-code data connectors and let us know what new connectors you want us to build next!