CSV Browser Upload
CSV Browser Upload
Quickly get started with Analytical Data Sharing by uploading CSV files directly from your browser. This feature allows you to create new tables in your Vendia Uni by uploading CSV files, configuring options, and defining column types. Great for fast prototyping without needing to set up complex data pipelines.
Note: The maximum file size you can upload via the browser is 2GB. Please use the Amazon S3 ingestion method for larger files.
Prerequisites
Before uploading CSV files, ensure that:
- Your CSV file is properly formatted with consistent data types per column
- The file size is under the 2GB limit for browser uploads
- Your data follows UTF-8 encoding for best compatibility
Supported File Formats
Currently, Vendia supports uploading CSV files with the following characteristics:
- Format: Comma-separated values (CSV)
- Encoding: UTF-8 (recommended) or other standard text encodings
- Maximum Size: 2GB per file
- Headers: Optional header row support
CSV Options
When uploading a CSV, you can customize the following options:
| Option | Description |
|---|---|
| Header | Specify if the CSV has a header row. |
| Delimiter | Specify the separator for values (e.g., comma, tab, semicolon). |
| Whitespace | Trim or preserve whitespace from the beginning and end of values. |
| Optional field enclosure | Specify if some values are wrapped in single or double-quotes. |
| Date format | Attempt to determine date format via auto-detection or manually specify date format. |
| Time format | Attempt to determine time format via auto-detection or manually specify time format. |
| Timestamp format | Attempt to determine timestamp format via auto-detection or manually specify timestamp format. |
| Error handling | Specify error handling actions (i.e., stop on error or skip errors). |
Column Configuration
After setting the CSV options, review and adjust the columns detected from your file:
- Column Name: The name of the column as detected from the CSV header or assigned automatically.
- Type: The data type for each column (e.g., INT, STRING, DATE).
- Sample Value: A sample value from your data to help verify correct type selection.
You can change the data type for each column as needed before completing the upload.
Supported Data Types
Vendia supports the following data types for CSV columns:
| Data Type | Description | Example Values |
|---|---|---|
| STRING | Text data of any length | ”John Doe”, “Product Name” |
| INTEGER | 32-bit integer numbers | 123, -456, 0 |
| LONG | 64-bit integer numbers | 1234567890123, -987654321 |
| FLOAT | Floating-point decimal numbers | 3.14, -2.5, 1.23E+10 |
| BOOLEAN | True/false values | true, false, 1, 0 |
| DATE | Date values | 2023-01-18, 1/18/2023 |
| TIMESTAMP | Date and time values | 2024-06-08 17:28:00 |
| BINARY | Binary data encoded as base64 or hex | base64 encoded data |
Configuration Tips
- Review sample values to ensure correct data type selection
- Adjust error handling to fit your data quality needs
Once you have configured all options and columns, proceed to upload your CSV to create the new table.
Best Practices
- Data Quality: Review sample values to ensure correct data type selection before uploading
- Error Handling: Adjust error handling settings to fit your data quality needs
- File Size: For files larger than 2GB, use the Amazon S3 ingestion method instead
- Encoding: Use UTF-8 encoding to prevent character encoding issues
- Headers: Include header rows in your CSV files for automatic column naming
- Data Types: Verify that each column’s data type matches the actual data content
- Testing: Start with smaller sample files to test configuration before uploading large datasets
Troubleshooting
If you encounter issues during CSV upload:
- File Size: Verify your file is under the 2GB limit
- Format Issues: Check that your CSV follows standard formatting conventions
- Encoding Problems: Ensure your file uses UTF-8 or another supported encoding
- Column Types: Review and adjust data types if automatic detection is incorrect
- Delimiter Detection: Manually specify delimiters if auto-detection fails
- Memory Issues: For very wide tables (many columns), consider splitting the data
- Special Characters: Check for unusual characters that might cause parsing errors
Next Steps
After successfully uploading your CSV file, you can:
- Query your newly created table using SQL
- Integrate the data with other tables in your Vendia Uni
- Create data products to share with data consumers