Now that we’ve built and filled our bucket with data, we want to bring it into Snowflake. Snowflake allows you to specify a file format with the copy command, meaning that whether my project utilizes JSON, CSV, Parquet or a mixture of all three, I can organize my data into a single S3 bucket for each project I am working on. Something I really like about the way Snowflake interacts with these S3 buckets is that the bucket can contain any of the supported file formats and Snowflake will allow you to specify what to pull out. To begin this process, you need to first create an S3 bucket (if you’re unfamiliar with this process, look here). Loading data into Snowflake from AWS requires a few steps: 1. The entire database platform was built from the ground up on top of AWS products (EC2 for compute and S3 for storage), so it makes sense that an S3 load seems to be the most popular approach. Loading from an AWS S3 bucket is currently the most common way to bring data into Snowflake. I’m going to spend a bulk of the time today talking about how to perform a simple AWS S3 load.

Since Snowflake has a multi-cloud architecture (Amazon Web Services, Microsoft Azure and a goal of Google Cloud support in the future), we luckily have a few options to get our tables loaded. Layered on top of the file formats are the protocols we can use to bring that data into Snowflake.

Snowflake supports a handful of file formats, ranging from structured to semi-structured. This week we’re going to talk about loading data into Snowflake, which due to its cloud nature, requires a different process than standard or legacy database systems. For this post, I want to talk about what happens before we can access the power of Snowflake with ANY data. Last month, I walked you through how to work with JSON in Snowflake and discussed the process Snowflake uses to flatten JSON arrays into a format that can be easily queried. For the second post in my continuing series on Snowflake, I wanted to expand on some concepts covered in my JSON post.

Create csv file format snowflake