# Agent Observability
Source: https://docs.upsolve.ai/ai-agent-builder/agent-observability
Inspect the step-by-step trace of how your agent answered a question, and the evaluations that graded it.
Every answer your agent produces is the result of a sequence of steps: retrieving schema, looking up golden examples, generating SQL, executing it, building a chart, and writing the narrative. Agent Observability gives you the full trace of that process — plus the automated evaluations that graded the result — so you can see exactly why an answer came out the way it did.
## Reaching a trace
**From a live chat** — after the agent responds, click **Observability →** beneath the answer. The trace opens in the right-hand panel and follows the conversation as it continues.
![A chat answer with the Observability link in its action row, and the trace panel opening on the right-hand side]()
**From Chat History** — open [Chat History](/ai-agent-builder/chat-history), click a row, and switch the sidebar to the **Observability** tab. This works for every conversation, including deployed end-user chats and MCP calls. The tab is hidden for conversations that made no tool calls, because there is nothing to trace.
![The Chat History detail sidebar with the Observability tab selected, showing the trace instead of the conversation]()
## What the trace shows
### The summary bar
At the top of the trace, a row of chips summarises the whole conversation:
* **steps** — total number of tool calls the agent made.
* **duration** — total time spent inside those calls, in milliseconds.
* **rag** — how many golden queries and golden charts were retrieved as references.
* **evals** — how many automated evaluations ran. The chip is green when all of them passed and amber when any did not.
* **errors** — present only when at least one step failed.
**Expand** and **Collapse** open or close every step at once.
![The trace view, with a summary chip bar of steps, duration, RAG sources and evals above the agent pipeline steps, the agent response and the evaluations]()
### Per-exchange sections
Below the summary, the trace is broken into one section per user question, each with up to four blocks:
**User Query** — the question as the user asked it.
**Agent Pipeline** — the ordered list of steps the agent took. Every step shows its own duration and success/error state, and expands to reveal its inputs and outputs. The steps you'll see include:
| Step | What it did |
| -------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Schema Retrieval** | Searched the data model for the tables and columns relevant to the question. |
| **Selectable Values** | Searched actual column values, so a filter on a category name resolves to a value that exists. |
| **Full Schema** | Loaded the whole data model — used for "what data do I have?" style questions. |
| **Golden Examples** | Retrieved confirmed question → SQL → chart pairs to use as references. |
| **SQL Generation** | Wrote the query. Shows the task it was given, the schema it was handed, the golden references it used, and the SQL it produced. If a number is wrong, start here. |
| **SQL Validation** | Checked the query against the database before running it for real. |
| **Data Retrieval** / **SQL Execution** | Ran the query and returned rows. Shows the row count and the columns that came back. |
| **Chart Generation** / **Render Chart** / **Chart Validation** | Chose the chart type, axis mappings and formatting. If the visualization is wrong but the numbers are right, the config here tells you what the agent picked. |
| **Knowledge Base Search** / **Knowledge Base File** | Searched or read a [Knowledge Base](/ai-agent-builder/knowledge-base) document. |
| **Skill** / **Skill Search** / **Skill Read** | Loaded a [Skill](/ai-agent-builder/skills) on demand. Internal plumbing skills are hidden; your own and learned skills are shown. |
| **Report Generation** / **Shareable Report** | Produced a CSV/PDF export or a shareable analysis. |
| **Suggestions** | Generated the follow-up questions offered under the answer. |
| **Learning Signal** | Recorded a correction the agent detected — the stage that went wrong, the failing artifact, and the user's correction. |
Tools you connected through [MCP](/ai-agent-builder/mcp-context) appear here too, under their own names.
**RAG Sources** — wherever a step used golden assets, the trace lists them as cards showing the retrieved question, its SQL, and a preview of its chart, each with the similarity score that earned it a place. When a trace shows the agent leaning on a bad example, the card names the [Golden Asset](/ai-agent-builder/golden-assets) to go and fix.
![The RAG Sources block in a trace, with the golden examples step expanded to list the SQL and chart references it retrieved, each card showing its similarity score]()
**Agent Response** — the narrative the user actually read.
**Evaluations** — the automated grades for this exchange. See below.
## Evaluations
Upsolve runs continuous, LLM-based evaluations against production traces. They run automatically in the background — you don't trigger them, and they don't slow the agent's answer down — and their verdicts flow back into the **Evals** column in [Chat History](/ai-agent-builder/chat-history) and into the **Evaluations** block of the trace.
### Reading a verdict
Each evaluation resolves to one of three states:
* **Pass** — the evaluation's criterion was met.
* **Fail** — the criterion was not met, or the evaluation itself errored.
* **Pending** — the evaluation hasn't returned a verdict yet. Evaluations run asynchronously, so a very recent conversation will often show pending for a short while.
The row-level verdict in Chat History is the worst state present: any failure makes the row fail, otherwise any pass makes it pass, otherwise it's pending. Hover the pill for the per-evaluation breakdown.
Expand an evaluation in the trace to see:
* **Score** — the numeric grade the evaluator returned, when it produced one.
* **Description** — what this evaluation checks.
* **Annotation** — the evaluator's written reasoning for the verdict it gave. This is the part worth reading: it usually names the specific thing that went wrong.
![The Evaluations block of a trace, listing each evaluation with its verdict, and one expanded to show its score, description and the evaluator's written annotation]()
### What gets evaluated
Evaluations fall into two broad families. The exact set is configured per deployment, so talk to your Upsolve contact to add one or tune an existing one for your agent.
**Answer quality** — did the agent actually answer the question, and is the answer right? *Answer Correctness Judge* is the workhorse here: it grades the response against the question that was asked and the data the agent retrieved to answer it.
**Conversation health** — how the exchange is going across turns, rather than within a single answer. *Multi-Turn Forgetfulness Detector* catches the agent losing context established earlier in the conversation — a filter the user set three turns ago that silently stopped being applied. *User Frustration Detector* catches the user re-asking, correcting, or pushing back, which is the earliest reliable signal that something is wrong even when no step errored and no thumbs down was given.
Conversation-health evaluations read the whole thread, so they only produce a meaningful verdict once a conversation has enough turns. A single-question session will typically show them as not applicable rather than pass or fail.
Some evaluations run purely for Upsolve's own quality monitoring and are deliberately not surfaced in the trace, so the count in the **evals** chip can be lower than the number of evaluations that actually ran.
## Using traces to improve your agent
Observability earns its keep when an answer is wrong. The common patterns:
**Wrong SQL, right-sounding answer** — the agent generated plausible SQL that doesn't implement your business logic. Open **SQL Generation** and read the query. The fix is usually a more precise [System Prompt](/ai-agent-builder/system-prompts) rule, or a [Golden Query](/ai-agent-builder/golden-assets) demonstrating the correct approach.
**No RAG sources retrieved for a common question** — the agent answered from scratch instead of referencing a validated example. Check whether a Golden Query covers this pattern, and whether the phrasing is semantically close enough to trigger retrieval.
**The wrong golden asset was retrieved** — the trace shows a RAG source that doesn't fit the question. Note which asset the card names, then sharpen its question text in [Golden Assets](/ai-agent-builder/golden-assets), or retire it.
**Correct SQL, wrong chart** — the data was right but the visualization misled. Add a [Golden Chart](/ai-agent-builder/golden-assets) for this question type.
**Errors in the pipeline** — a red step names the failure directly. A failed SQL execution usually points at a data model that promises a column the warehouse doesn't have; a failed validation usually points at generated SQL the dialect rejects.
**High cost or many steps** — the trace shows repeated SQL attempts before a correct result. That normally means the data model annotations or system prompt need more specificity, so the agent stops guessing.
**A thumbs down with no obvious error** — read the **Annotation** on the failing evaluation, then the **Agent Response** block. Framing problems (over-claiming, missing caveats, too much jargon) don't surface as errors anywhere else.
## Next steps
Browse conversations as sessions or traces, and filter on evals, feedback, errors and cost.
Add example queries and charts to fix patterns you identify in traces.
# Agents
Source: https://docs.upsolve.ai/ai-agent-builder/agents
Build AI chat experiences that understand your data.
## What is an Agent?
An Agent is an AI-powered chat interface that:
* Converts natural language questions into SQL
* Analyzes results and provides insights
* Creates visualizations from query results
* Learns from examples (Golden Assets)
![Agent chat interface]()
## Agent Components
### Data Model Link
Every agent is linked to a specific data model version. This ensures:
* The agent knows what tables and columns exist
* Changes to the data model don't break the agent
* You can test with different data model versions
### System Prompt
A custom prompt that shapes how the agent responds:
* Company context and terminology
* Response style preferences
* Special instructions
### Golden Assets
Example queries and charts that teach the agent:
* How to write SQL for common questions
* What visualizations to create
* Correct terminology and calculations
### Evaluations (Evals)
Test sets to verify agent accuracy:
* Known question-answer pairs
* SQL validation
* Result comparison
## Creating an Agent
1. Navigate to your project's **Agents** tab
2. Click **Create Agent**
3. Enter a name and description
4. Select the data model to use
5. Click **Create**
![Create agent dialog]()
## Configuring Your Agent
### Setting the System Prompt
1. Open your agent
2. Go to the **System Prompt** tab in the sidebar
3. Write instructions for the agent
4. Click **Save**
Example system prompt:
```
You are a data analyst for an e-commerce company.
Use clear, business-friendly language.
When users ask about "revenue," they mean gross_sales minus refunds.
Always include date ranges in your analysis.
```
![System prompt editor]()
### Adding Golden Assets
Golden Assets are example queries that improve accuracy:
1. Go to the **Assets** tab
2. Click **Add Asset**
3. Enter the question and corresponding SQL
4. Optionally add a chart configuration
5. Click **Save**
![Golden assets list]()
**Tip:** The more golden assets you add for common questions, the more consistent your agent becomes.
### Creating Evaluations
1. Go to the **Tests** tab
2. Click **Add Test**
3. Enter a question
4. Enter the expected SQL or answer
5. Save and run the test
![Agent test interface]()
## Admin View vs User View
### Admin View
As an admin, you see full details:
* Every tool call the AI makes
* SQL queries generated
* Step-by-step reasoning
* Token usage and timing
![Admin chat view with tool calls expanded]()
### User View
End users see a clean interface:
* Just the question and answer
* Charts and insights
* No technical details
![User chat view]()
## Versioning
Like data models, agents are versioned:
1. Go to the **Versions** tab
2. See all versions with timestamps
3. Each version tracks: assets, prompt, data model link
![Agent versions list]()
## Setting Production
Before users can access your agent, you must set a version as production.
### Requirements
1. The agent must be linked to a data model
2. That data model version must already be production
3. The agent's schema requirements must fit within the data model
### Steps
1. Open your agent
2. Go to the **Versions** tab
3. Click **Set as Production** on the version you want
4. The system validates everything
5. If successful, users can now access this version
![Set agent as production]()
### Validation Errors
If validation fails, you'll see specific errors:
* "Data model version is not production" - Set the data model production first
* "Missing tables/columns" - The agent uses data not in the data model
## Changing Data Model Version
To link your agent to a different data model version:
1. Go to the **Data Model** tab in the sidebar
2. Select the data model
3. Choose a version (production versions recommended)
4. Click **Save**
This creates a new agent version with the new data model link.
![Change data model version]()
## Best Practices
### 1. Start with Golden Assets
Add 10-20 golden assets covering common questions before going live.
### 2. Use Descriptive Data Models
Column descriptions in your data model help the agent understand your schema.
### 3. Run Evals Regularly
Create a test suite and run it whenever you update the agent.
### 4. Monitor in Admin View
Periodically check admin view to see how the agent handles real questions.
## Next Steps
* [Build an Application](/ai-agent-builder/applications) to combine agents with dashboards
* [Set up the complete flow](/ai-agent-builder/setup-guide) from project to production
# Applications & Spaces
Source: https://docs.upsolve.ai/ai-agent-builder/applications
Build dashboard templates and deploy them to customer spaces.
## What is an Application?
An Application is a container for dashboard templates that you deploy to your customers. It includes:
* **Chart Templates** - Reusable chart configurations
* **Dashboard Templates** - Layouts combining multiple charts
* **Spaces** - Customer-specific instances of your templates
* **Default Settings** - Language, timezone, export options
![Application with templates and spaces]()
## How Applications and Spaces Work
```mermaid theme={null}
flowchart TD
A[Application] --> B[Chart Templates]
A --> C[Dashboard Templates]
A --> D[Spaces]
D --> E[Space for Org 1]
D --> F[Space for Org 2]
D --> G[Space for Org 3]
```
Each space receives its own private copies of your published templates.
When you publish a template:
1. It becomes available to all spaces
2. Each space gets its own private copy
3. Users can customize their copies without affecting others
## Creating an Application
1. Navigate to your project's **Applications** tab
2. Click **Create Application**
3. Enter a name
4. Select a data model
5. Click **Create**
![Create application dialog]()
## Building Templates
### Creating a Chart Template
1. Open your application
2. Click **Add Chart**
3. Choose a chart type
4. Configure the data and visualization
5. Click **Save**
![Chart builder interface]()
### Creating a Dashboard Template
1. Open your application
2. Click **Add Dashboard**
3. Drag and drop charts onto the canvas
4. Arrange the layout
5. Add filters if needed
6. Click **Save**
![Dashboard builder interface]()
## Viewing as a User
While building templates, you see them as an admin. To see what users will see:
1. Select a project user from the dropdown
2. The view switches to their perspective
3. Data is filtered by their RLS rules
4. Check that the template works for different users
![User impersonation dropdown]()
## Publishing Templates
Templates start in "draft" mode. To make them available to users:
1. Open the template
2. Click **Publish**
3. The template propagates to all spaces
![Publish button on template]()
### What Happens When You Publish
1. Template status changes to "published"
2. Each space gets a copy of the dashboard
3. Users can now see and interact with it
4. Future updates require re-publishing
## Understanding Spaces
Each project organization automatically gets a Space for each application.
### What's in a Space?
* **Template Copies** - Private instances of published templates
* **User Dashboards** - Dashboards users create themselves
* **User Charts** - Charts users create themselves
### Viewing Spaces
1. Open your application
2. Go to the **Spaces** tab in the sidebar
3. See all spaces (one per organization)
4. Click to preview any space
![List of spaces in application]()
## Default Views
You can set which dashboard users see first when they open their space.
### Setting an Application Default
1. Open your application
2. Go to the **Templates** tab
3. Click the star icon next to a template
4. This becomes the default for all users
![Setting default template]()
### User-Specific Defaults
Users can change their own default:
1. Open any dashboard in their space
2. Click **Set as Default**
3. This overrides the application default for them only
## Application Settings
### Default Language
Set the default language for your application:
1. Go to application settings
2. Select default language
3. Users can override in their preferences
### Default Timezone
Configure how dates are displayed:
1. Go to application settings
2. Set the default timezone offset
3. Enable "Use Local Timezone" for auto-detection
### Export Settings
Control whether users can export data:
1. Go to application settings
2. Toggle "Allow Exports"
![Application settings panel]()
## Production Status
Like data models and agents, applications can be set to production.
### Requirements
1. Application must have a linked data model
2. That data model version must be production
3. All template schemas must fit within the data model
### Setting Production
1. Open your application
2. Go to the **Versions** tab
3. Click **Set as Production**
4. Validation runs automatically
## Connecting Agents and Applications
If an agent and application share the same data model version:
* The agent can generate charts compatible with the application
* Users can save agent-generated charts to their space
* Charts flow seamlessly between AI and dashboards
## Best Practices
### 1. Test with Multiple Users
Always preview templates as different users to ensure RLS works correctly.
### 2. Start with Core Dashboards
Create 2-3 essential dashboards before adding more.
### 3. Use Consistent Naming
Name templates clearly so users understand what they show.
### 4. Document in Descriptions
Add descriptions to dashboards explaining what they display.
## Next Steps
* Follow the [Complete Setup Guide](/ai-agent-builder/setup-guide) for the full workflow
* Learn about [Row-Level Security](/ai-agent-builder/rls) for data filtering
# Chat History
Source: https://docs.upsolve.ai/ai-agent-builder/chat-history
Review every conversation your agent has handled, as sessions or as individual traces.
Chat History is the operational view of your agent. It lists every conversation the agent has handled — sessions run in Agent Studio, live chats from deployed end users, and calls that came in over MCP — with the quality, cost and reliability signals for each one attached to the row.
It answers two different questions, and it has a view for each:
* **Sessions** — "how is this conversation going?" One row per conversation.
* **Traces** — "how did this answer turn out?" One row per user question → agent answer exchange.
Use the **Sessions / Traces** toggle in the toolbar to switch. Both views share the same filters, column controls, time range and CSV export; only the grain changes.
## Sessions view
One row per conversation. Available columns:
| Column | What it shows |
| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| **Date** | When the conversation started. |
| **Evals** | Aggregate verdict of the automated evaluations that ran on this conversation — pass, fail, or pending. Hover for the per-eval breakdown. |
| **Summary** | A short generated description of what the conversation was about. |
| **Feedback** | The thumbs up / thumbs down the user submitted, if any. Hover to read the written comment. |
| **Charts** | How many charts the agent built. |
| **Errors** | How many steps failed — a failed SQL execution, a tool error, a rejected query. |
| **Questions** | How many questions the user asked in the conversation. |
| **RAG Sources** | How many golden queries and golden charts were retrieved as references. |
| **Cost** | Model spend for the conversation, in USD. |
| **Duration** | Wall-clock length of the session. |
| **User** | The project user who was chatting, or *Admin* for a builder session in Agent Studio. |
| **Platform** | Where the conversation came from — *Agent Studio*, *Embedded*, or *MCP*. |
**Duration**, **User** and **Platform** are hidden by default. Turn them on from the column picker.
![The Chat History table in Sessions mode, one row per conversation, with columns for evals, summary, feedback, charts, errors, questions, RAG sources and cost]()
## Traces view
One row per exchange, so a five-question conversation produces five rows. This is the grain to work at when you're triaging quality: an eval failure, a thumbs down or an error belongs to a specific answer, not to the whole session.
| Column | What it shows |
| ---------------- | ------------------------------------------------ |
| **Time** | When the exchange happened. |
| **Evals** | The evaluation verdict for this specific answer. |
| **User Input** | The question as the user asked it. |
| **Agent Output** | The answer the agent gave. |
| **Feedback** | Thumbs up / down on this answer. |
| **Charts** | Charts produced by this answer. |
| **Errors** | Failed steps within this answer. |
| **RAG Sources** | Golden assets retrieved for this answer. |
| **Cost** | Model spend attributed to this exchange. |
| **User** | Who asked. |
| **Platform** | Where the question came from. |
| **Trace ID** | The observability trace identifier. |
| **#** | The exchange's position within its conversation. |
**User**, **Platform**, **Trace ID** and **#** are hidden by default.
![The Chat History table in Traces mode, one row per user question and agent answer, with the user input and agent output shown inline]()
## Working the table
**Filters** — the chip bar on the left of the toolbar filters on any visible column: eval status, feedback, platform, user, and numeric ranges for charts, errors, questions, RAG sources and cost. Filters follow column visibility: hiding a column hides its filter but keeps the filter's value, so toggling the column back on restores it.
**Time range** — narrow to the past 5 minutes, 30 minutes, hour, day, week, month, 3 months or year. Defaults to all time.
**Columns** — the column picker lets you show and hide columns and drag them into the order you want. Columns can also be resized by dragging their edge in the header. Your choices persist per view; **Reset to defaults** in the picker restores the shipped layout.
**Reload** — re-fetch without leaving the page. Useful when you're watching a live deployment.
**Export CSV** — exports the current view with the columns you have visible, so the export matches what you're looking at.
![The Chat History toolbar with the column picker open, showing each column as a draggable checkbox row, alongside active filter chips and the time-range picker]()
## Opening a conversation
Click any row to open the detail sidebar.
The metadata card at the top carries **Date**, **Duration**, **Tenant**, **Agent**, **Platform**, **Cost**, the **Session ID**, and — when you opened the sidebar from a Traces row — the **Trace ID** for that exchange. Both identifiers can be copied with one click. If the user left feedback, **Feedback** and their **Comment** appear underneath.
Below the metadata card are two tabs:
* **Chat** — the conversation exactly as it happened, including the charts the agent rendered.
* **Observability** — the step-by-step trace of what the agent did. This tab only appears for conversations that actually made tool calls.
![The Chat History detail sidebar open on a conversation, with the metadata card at the top and the Chat and Observability tabs below it]()
Everything here covers deployed end-user chats as well as your own Agent Studio sessions. There is nothing extra to enable — a conversation from an embedded deployment or an MCP client shows up with its **Platform** set accordingly.
## Next steps
Inspect the step-by-step trace behind any answer, and the evaluations that graded it.
Add example queries and charts to fix patterns you identify in past conversations.
# Data Model & Schema
Source: https://docs.upsolve.ai/ai-agent-builder/data-model-schema
Choose which tables and columns the agent can see, and encode what they mean.
The Data Model is the foundation of your agent's knowledge — every other entry point in this section ([System Prompts](/ai-agent-builder/system-prompts), [Golden Assets](/ai-agent-builder/golden-assets), [Skills](/ai-agent-builder/skills), [Knowledge Base](/ai-agent-builder/knowledge-base), [User Memory](/ai-agent-builder/user-memory), and [MCP](/ai-agent-builder/mcp-context)) builds on top of it. It controls two things: what data the agent can access, and what context it has to interpret that data correctly. A well-configured data model is the difference between an agent that writes plausible SQL and one that writes the *right* SQL for your business.
## Selecting tables and columns
Not every table in your database is relevant to every question your agent will answer. Narrowing the schema down to what matters reduces ambiguity, improves query accuracy, and avoids the agent making inferences across data it shouldn't touch.
![Table selection panel showing checked and unchecked tables]()
From the **Tables** tab in your Data Model, select the tables you want to expose to the agent. By default all tables from your connection are listed — check only the ones that are relevant to the use case you're building for.
Within each table, you can further control which columns are included. Expand a table to see its columns, then uncheck any that are unnecessary, sensitive, or likely to confuse the agent. The column list shows the name, data type, and any description you've added.
Less is more here. An agent scoped to 5 relevant tables with clear descriptions will significantly outperform one with access to 50 tables it has to guess about.
## Adding table and column descriptions
Descriptions are how you encode institutional knowledge directly into the schema. The agent reads these when generating SQL — they're the mechanism for telling it what your data actually means, not just what it's called.
![Column list showing descriptions and selectable toggles]()
**Table descriptions** provide context for the whole table — what it represents, how it's populated, any important caveats. Click into a table and use the **Table Description** field at the top of the column panel to add this.
**Column descriptions** sit inline next to each column. Click the description field for any column and add a plain-English explanation. Useful descriptions answer the question a developer would have when first looking at the column: what does this value represent, what are the units, is there anything non-obvious about how it's calculated?
For example:
* `contact_name` → *"Name of the customer's point of contact for this order"*
* `owner_name` → *"Name of the sales rep (seller) who owns this account"*
* `arr` → *"Annual Recurring Revenue in USD, calculated as MRR × 12. Excludes one-time fees."*
The more specific your descriptions, the more reliably the agent will apply the right column to the right question.
## Marking columns as selectable
The **Selectable** toggle controls whether the agent can scan a column's distinct values at query time to inform how it builds SQL.
![Column row showing the Selectable toggle in the on and off states]()
When a column is marked selectable, the agent can run `DISTINCT()` over it to extract its unique values, and then use those values in two ways:
* **Categorical filtering in SQL writing** — the agent knows the valid filter values *before* it writes a `WHERE` clause. It knows that `region` contains "North America", "EMEA", and "APAC", so it filters on real values rather than guessing at spellings or inventing categories.
* **Text grep search** — the agent can search across a selectable column's values to find the exact entry a user referred to loosely (e.g. matching "acme" to the `customer_name` value "Acme Corp (US)").
Think of it as giving the agent the ability to power a select-style filter: it knows the possible values and can use them precisely.
Mark a column as selectable when:
* It contains categorical values users would naturally filter by (region, product, owner, status)
* The distinct values are finite and meaningful to business questions
* You want the agent to be able to enumerate or search options rather than guess at valid values
Leave it off for high-cardinality columns (IDs, free-text fields, timestamps) where scanning distinct values would be noisy or meaningless.
## Beyond the schema: the Knowledge Base tab
The Data Model editor has three tabs — **Tables**, **Global Data Security**, and **Knowledge Base**. Descriptions on tables and columns cover what your *data* means, but some context isn't about any one column: business policies, company background, definitions of terms users use loosely, and known caveats about the data. Those live in the Knowledge Base as searchable markdown documents that ship with each data model version. See [Knowledge Base](/ai-agent-builder/knowledge-base) for how to add and organize them.
## Saving your changes
Changes to the data model create a new draft version. Click **Save Version** in the top-right corner when you're ready to checkpoint your work. A data model version must be marked as **Production** before an agent can use it in a live environment.
Agents are linked to a specific data model version. Updating the data model does not automatically update agents that reference it — you'll need to update the agent's data model link and set a new production version.
## Next steps
Encode business rules, terminology, and behavioral guardrails into the agent.
Add example queries and charts the agent can reference when answering similar questions.
Store reference documents the agent searches on demand to interpret your terms.
Package procedures and conventions the agent loads when it needs them.
Let the agent record and apply each end user's preferences automatically.
Connect external tools and live systems as additional agent context.
# Data Models
Source: https://docs.upsolve.ai/ai-agent-builder/data-models
Create curated views of your database with descriptions, annotations, and versioning.
## What is a Data Model?
A Data Model is a curated view of your database that:
* **Selects** which tables and columns are available
* **Describes** fields with human-readable annotations
* **Secures** data with row-level security rules
* **Versions** changes so you can track and rollback
![Data model showing selected tables and columns]()
## Why Use Data Models?
1. **Simplify for AI** - Only expose relevant tables so AI agents aren't overwhelmed
2. **Add context** - Descriptions help AI understand what columns mean
3. **Control access** - Data Security rules ensure users see only their data
4. **Safe deployments** - Versioning prevents breaking changes
## Creating a Data Model
1. Navigate to your project's **Data Models** tab
2. Click **Create Data Model**
3. Select the connection to base it on
4. Enter a name for your data model
5. Click **Create**
The system will fetch all tables and columns from your connection automatically.
![Create data model dialog]()
## Configuring Your Data Model
### Selecting Tables and Columns
By default, all tables are selected. To customize:
1. Open your data model
2. Uncheck tables you want to hide
3. Expand tables to uncheck specific columns
4. Click **Save Changes**
![Table and column selection interface]()
### Adding Descriptions
Help AI and users understand your data:
1. Click the edit icon next to a table or column
2. Enter a description
3. For AI agents, mark columns as "selectable" if they contain categorical values users might filter by
![Adding description to a column]()
## Data Security
Data Security controls which rows of data each user can access. You can define **Global Security Rules** that apply across all tables, or configure per-table rules for specific needs.
### Understanding Data Security
The Data Security system uses SQL-based filtering with dynamic variables:
* **`{{user.*}}`** - Access properties of the currently authenticated user
* **`{{organization.*}}`** - Access properties of the user's organization
For example, to filter an `orders` table so users only see their own orders:
```sql theme={null}
SELECT *
FROM "orders"
WHERE "user_id" = '{{user.id}}'
```
### Global Security Rules
Global Security Rules let you define filtering logic once and apply it to all applicable tables automatically. This is ideal when you have consistent patterns like:
* Multi-tenant data separated by `tenant_id`
* User-owned data filtered by `user_id`
* Schema-based isolation (each tenant has their own database schema)
#### Accessing Global Rules
1. Open your data model
2. Click the **Global Data Security** tab, alongside **Tables** and **Knowledge Base**
3. You'll see the Global Rules editor, with **+ Add Rule** above any rules that already exist
#### Rule Types
**Schema Rules** - Dynamic schema prefixing for multi-tenant databases where each tenant has their own schema:
![The Schema Rule form with Schema Property set to user.source, and an Example Output line showing every query prefixed with that schema]()
This generates SQL like:
```sql theme={null}
SELECT *
FROM "{{user.schema}}"."orders"
```
**Column Rules** - Automatic WHERE clause filtering based on column names:
![The Column Rule form filtering the country column on user.country, with the Optional Rule toggle on, a Tables Affected summary, and the generated WHERE clause]()
This generates SQL like:
```sql theme={null}
SELECT *
FROM "orders"
WHERE "user_id" = '{{user.id}}'
```
#### Creating a Global Rule
1. Click the **Global Data Security** tab
2. Click **Add Rule**
3. Choose **Schema Rule** or **Column Rule**
4. Configure the rule settings
5. The rule is automatically enabled
![The Add Global Rule dialog offering Schema Rule and Column Rule, each with a one-line description]()
#### How Rules Are Applied
* **Schema rules** affect all tables - they change where data is fetched from
* **Column rules** only affect tables that have the specified column - tables without the column are unaffected
* Multiple rules are combined with AND logic
* Rules are applied in priority order (lower priority number = applied first)
Applying the rules opens a preview first: the SQL each table will be given, a count of how many will be filtered, and — because a manual override always wins — how many will be skipped.
![The Apply Global RLS Rules preview, listing the SQL each table will be given, with a count of the tables to be filtered and those skipped for having a manual override]()
### Per-Table Data Security
Selecting a table under **Tables** gives it its own **Data Security** sub-tab — distinct from the model-wide **Global Data Security** tab — showing how the global rules apply to that one table.
![A table's Data Security sub-tab managed by the global rules, its Applied Rules list empty because this table has no column the rules match, above the read-only generated query]()
#### Manual Overrides (Break Glass)
Sometimes you need custom logic for a specific table. You can override global rules:
1. Select the table under **Tables**
2. Go to its **Data Security** sub-tab
3. Click **Edit manually**
4. Confirm the warning dialog
5. Edit the SQL directly
![The Switch to Manual Security dialog, listing that the generated SQL is copied to the editor, the table stops inheriting global rules, and later global changes will not reach it]()
Once in manual mode, you have full control over the SQL:
![The same tab in manual override mode: an amber manually-edited banner with Reset to global, an editable query, and the user and organization properties available to it]()
#### Resetting to Global Rules
To return a table to global rule management:
1. Click **Reset to global**
2. Confirm the dialog
3. Your manual edits will be replaced with the auto-generated SQL
![The Reset to Global Security dialog, listing that the manual SQL is deleted, the global rules are re-applied, and later global changes will affect the table again]()
### Testing Data Security
You can test your security rules to see exactly what data a specific user would see:
1. Select a table under **Tables**
2. Go to its **Data Security** sub-tab
3. Select a user from the top-right user dropdown
4. Click **Test Query**
5. View the filtered results
![Test RLS Policy run as a supplier manager whose user.country resolves to Canada and Mexico, showing the query sent to the database and the 18 rows it returned]()
Testing uses your actual database connection and shows real data filtered by the security rules. This helps verify your configuration before deploying.
## Versioning
Every change to a data model creates a new version. This means:
* You can see the history of changes
* You can compare versions
* Agents and applications link to specific versions
### Viewing Versions
1. Open your data model
2. Click the **Versions** tab in the sidebar
3. See all versions with timestamps and change information
![Data model versions list]()
## Production Status
Before agents and applications can use a data model in production, you must mark a version as "Production."
### Setting Production
1. Open your data model
2. Go to the **Versions** tab
3. Click **Set as Production** on the version you want
4. The system validates the schema against your connection
5. If successful, the version is now production
![Set version as production button]()
### Validation
When setting production, the system checks:
* All tables in your data model exist in the connection
* All columns in your data model exist in their tables
* If validation fails, you'll see which tables/columns are missing
If your database schema changes, you may need to update your data model to match.
## Syncing with Connection Changes
If your database schema changes:
1. Open your data model
2. Go to the **Connection** tab
3. Click **Refresh Schema**
4. New tables/columns will appear (you can select them)
5. Missing tables/columns will show warnings
![Sync with connection showing new and missing items]()
## Data Model Errors
If a production data model has errors (e.g., missing tables), you'll see a warning banner. This typically happens when:
* A table was dropped from the database
* A column was renamed or removed
* The connection credentials changed
## Next Steps
* [Create an Agent](/ai-agent-builder/agents) that uses this data model
* [Build an Application](/ai-agent-builder/applications) for dashboards
* [Manage Project Users](/ai-agent-builder/users) to define user properties for Data Security
# Backend Setup
Source: https://docs.upsolve.ai/ai-agent-builder/deploy-agents/backend-setup
Set up user authorization in the backend for agent deployment
## User Authorization
The purpose of user authorization set up in the backend is for Upsolve AI to provide appropriate data access based on your user permissioning set up.
In your product's authentication flow, you register users with Upsolve using the [project user registration flow](/embedded-bi/data-permissioning/project-user-registration-flow): register an [organization](/api-reference/endpoint/register-project-organization) once per customer entity, register a [user](/api-reference/endpoint/register-project-user) once per account, then fetch a short-lived [user token](/api-reference/endpoint/get-project-user-token) on every login. The token tells Upsolve AI who the authorized user is and what data they can access, and is used to provide the appropriate access to the agent and underlying data.
To generate your API Key please go to the deploy page.
## Project User Token
When embedding an Application Space (via the `https://ai-hub.upsolve.ai/share/application/:applicationId` iFrame), you need a **project user token**. This token identifies a specific user within a project organisation and is used by the embed to load their personalised Space.
### Endpoint
```
POST https://api.upsolve.ai/v1/api/projects/user-token
```
### Request
| Field | Type | Required | Description |
| ---------------- | ------------- | -------- | --------------------------------------------------------------------- |
| `userId` | string (UUID) | Yes | The project user's ID (found in your project's user list) |
| `organizationId` | string (UUID) | Yes | The organisation ID the user belongs to |
| `apiKey` | string | No | Your Upsolve embed API key (if not passed via `Authorization` header) |
| `expiration` | number | No | Token lifetime in seconds (default: 3600) |
### Response
```json theme={null}
{
"data": {
"token": ""
}
}
```
### Example
```typescript theme={null}
const response = await fetch("https://api.upsolve.ai/v1/api/projects/user-token", {
method: "POST",
headers: {
"Content-Type": "application/json",
"Authorization": "Bearer up_embed_...", // Your embed API key
},
body: JSON.stringify({
userId: "user-uuid-here",
organizationId: "org-uuid-here",
}),
});
const { data } = await response.json();
const projectUserToken = data.token;
```
Pass `projectUserToken` as the `jwt` query parameter in the Application iFrame `src`. See [Frontend Setup](/ai-agent-builder/deploy-agents/frontend-setup) for the full embed example.
Project user tokens expire after 1 hour by default. Refresh them server-side before they expire and update the iFrame `src` to avoid session interruptions.
New integrations should use `POST /v1/api/projects/user-token` directly. Legacy tenant-based integrations remain supported, but the project user flow is the recommended path going forward.
## Setup Inspection
You could inspect whether the user authorization is setup successfully. Navigate to the **Deploy** application using the side navigation bar.
![upsolve-deploy]()
If the endpoint is successfully called, you should see your new tenants in the **Deploy** application:
![tenants]()
# Canvas Iframe Embed
Source: https://docs.upsolve.ai/ai-agent-builder/deploy-agents/canvas-iframe-embed
Embed a public canvas into your product with an iframe
## Overview
You can embed a public Agent Canvas into your own product using a standard HTML `
```
* Replace `{accessToken}` with the token from your share modal.
* `?hideHeader=1` hides the canvas name and "Read-only" badge for a chrome-free embed (see [URL Parameters](#url-parameters)).
* For on-premise deployments, replace `ai-hub.upsolve.ai` with your instance's hostname.
That's it. The `/shared/canvas/:token` route sends `Content-Security-Policy: frame-ancestors *`, so the page can be embedded from any origin. Your users see **your** app's page in the address bar; the canvas renders inside the frame. You do **not** need a custom domain or any DNS changes for this.
## (Optional) White-label the share link with your own domain
By default the share modal generates links against `ai-hub.upsolve.ai`. If you'd rather the copyable link show your own domain, set a **Custom Share Domain**:
1. Go to **Project Settings → Sharing**.
2. Enter your **Custom Share Domain** — the base that replaces the `ai-hub.upsolve.ai/shared/canvas` portion of the link, e.g. `insights.yourcompany.com/canvas`. A live preview shows the resulting link.
3. Save.
From this point on, every canvas's public link resolves to `https:///` — e.g. `https://insights.yourcompany.com/canvas/{accessToken}`.
The Custom Share Domain **only rewrites the link string** Upsolve generates — it does not change where the canvas is served. You decide what that URL does:
* **Serve it from your own app (no DNS setup).** Route that path in your product (e.g. `/canvas/:token`) to a page you build that iframes the Upsolve canvas — the token is right there in the URL, so your page just renders `
```
For **Select Multi** filter types, the value **must** be a JSON array, not a
plain string. For example `filter_region=%5B%22North%20America%22%5D` (which is
`["North America"]` URL-encoded). Plain strings cause chart load errors.
### Multiple Controlled Filters
Combine multiple `filter_` parameters in the same URL:
```html theme={null}
```
For the full URL parameter reference — including omnibar (`f_`) filters and combining both types — see [URL Filter Parameters](/embedded-bi/filters/url-filter-parameters).
## Visibility in Embedded Contexts
By default, controlled filters are **hidden** when your dashboard is embedded, keeping them invisible to end users.
If you need to show a controlled filter in embedded contexts (such as admin panels or internal tools), set **`isVisible: true`** on the filter's control configuration in the Hub. The filter will then render as a visible pill while still being driven by your `filter_` URL parameter.
## Best Practices
1. **Use Descriptive Keys**: Choose filter keys that clearly indicate their purpose (e.g., `userRegion`, `departmentFilter`)
2. **Document Your Filters**: Keep track of which filters are controlled and their expected values
3. **JSON-encode values**: Always JSON-encode (and URL-encode) the value, especially arrays for Select Multi filters
## Troubleshooting
### Controlled Filter Not Working
* Verify the filter key matches exactly between the Hub configuration and your `filter_{controlKey}` URL parameter
* Check that the filter value type matches the filter configuration (string, number, array, etc.)
* For Select Multi filters, ensure the value is a JSON-encoded array (`%5B%22value%22%5D`), not a plain string
### Controlled Filter Hidden When It Should Be Visible
* Set `isVisible: true` on the filter's control configuration in the Hub to override the default hiding behavior
# Filtering
Source: https://docs.upsolve.ai/embedded-bi/filters/filters-in-depth
An in depth guide for setting up filters
## Overview
Dashboards in Upsolve are **source agnostic**. This means that multiple charts backed by multiple data sources may co-exist on the same dashboard. This provides endless flexibility to dashboard builders, but also requires filtering to be built on a set of rails that is different from traditional BI tools.
The basic concept is that there are [chart filters](#chart-filters) and [dashboard filters](#dashboard-filters). A **dashboard** filter can only be configured to point to one or more **chart** filters. This ensures a consistent behavior for the chart, whether it is being viewed/filtered as a standalone experience, or as part of a dashboard.
## Chart filters
To create a chart filter, go into the chart editor and select the `Configure filters` tab.
![Filter Guide 1]()
There are 3 things you can filter on:
**1. Raw backing tables:** filter any column in the raw data that is powering this specific chart
**2. Computed table:** filter any column from the data table that was computed by the SQL written for this chart
**3. Variables in the chart SQL:** inject variables into the SQL code that is backing this chart
![Filter Guide 2]()
There are 3 steps to configuring a filter:
**1. Table:** select the table to filter or "Query variables"
**2. Column:** select the column or variable to filter on
**3. Filter:** configure the filter as needed. Options will be provided based on the type of filter selected
![Filter Guide 3]()
Once the filter is added, navigate back to the `Chart Builder` tab to check it out. Click the blue filter icon to see all available filters. Setting one will add the filter to your chart
![Filter Guide 4]()
If you are happy with your result, `Save` the chart and let's configure this in the dashboard!
## Dashboard filters
To create a dashboard filter, go into the dashboard editor and click the `+ New filter` button.
![Filter Guide 5]()
Give your filter a name, and set the type and subtype. In this case, we are going to set a `Select` filter with multiple options allowed.
![Filter Guide 6]()
Note that there are only 2 charts on this dashboard that have `Select` filters avaialble for us to map to. Select the ones that this dashboard filter should map to. In this case, it will be `Expense Status 2` in both `Total Spend ($) (Copy)` and `Daily Spend Trend (Copy)`. Click the **green (+) button** next to each filter you want to target and you will see this:
![Filter Guide 7]()
![Filter Guide 8]()
Now, there are more expense statuses I want to map to, but the charts that are in this dashboard don't yet have those filters created on them. We can add them from this menu. Click the `# charts with no Select filter shown` button to reveal the other charts. Pick the one you want, such as `Spend by Merchant (Copy)`. Configure it to be a `Select` filter and preview to make sure things worked.
![Filter Guide 9]()
Add the filter and hit `Save` to make sure it's added. You will see the filter show up above the panel you are in with a green **(+)** icon. Note that if this filter was not configured as a `Select` filter, it would not appear in this case. This is because chart filters are only shown if they match the dashboard filter type exactly.
![Filter Guide 10]()
You may now click the **green (+) icon** to add this newly configured chart filter to the target list of this filter
![Filter Guide 11]()
Finally, use the filter preview to confirm that everything is working as expected.
![Filter Guide 12]()
Ther filter may now be set on the dashboard level. Note that all charts that the filter is connected to are filtered.
![Filter Guide 13]()
# Timezones
Source: https://docs.upsolve.ai/embedded-bi/filters/timezone
Configure timezones for your dashboard
## Overview
Dashboards provide flexible timezone configurations to ensure your data is displayed in a way that best suits your end users' needs. By default, dashboards automatically adjust to the **local timezone of the end user**, contextualizing data for their location.
## Key Requirements
* **Data Storage in UTC:** Ensure all timestamps in your data source are stored in **UTC**. Upsolve automatically adjusts these timestamps to match the configured timezone.
* **Automatic Adjustment:** Date columns will automatically shift to reflect the end user's local timezone unless otherwise specified.
## Setting a Specific Timezone
If you need all end users to see data in the same timezone, regardless of their location, you can configure a specific timezone through the **dashboard builder settings**.
### Steps to Set a Timezone:
1. Open the **Settings** dropdown in the dashboard builder.
2. Select the desired timezone from the available options.
3. To revert to displaying data in the end user's local timezone, select the **Local** option.
By setting a specific timezone, your dashboard provides a consistent view of time-sensitive data for all users.
![Timezone light]()
![Timezone dark]()
## Notes
* Dashboards always assume data is stored in **UTC** and adjust based on the selected timezone.
* Switching back to the **Local** option ensures data reflects the end user's local timezone automatically.
# URL Filter Parameters
Source: https://docs.upsolve.ai/embedded-bi/filters/url-filter-parameters
Pre-filter shared dashboards via URL query parameters
## Overview
When embedding dashboards in iframes or sharing application links, you can pre-set filter values directly in the URL. This lets you:
* Show different data views to different users without building separate dashboards
* Deep-link to a specific filtered state
* Control filters programmatically from your host application
## Omnibar Filters (`f_` prefix)
The primary way to pass filter values via URL. These filters show as interactive pills in the filter bar — users can see what's filtered and modify or remove them.
### URL Format
```
f_{sourceType}.{tableName}.{columnName}={typePrefix}:{value}
```
**Source types:**
* `Raw` — a column from a raw database table
* `Output` — a computed/output column from chart SQL
* `Variable` — an RLS variable filter
### Value Types
| Prefix | Type | Example |
| ------ | --------------------- | ----------------------------------------- |
| `s:` | Select (single value) | `s:Athletics` |
| `sm:` | Select (multi-select) | `sm:Athletics` or `sm:Athletics,Swimming` |
| `n:` | Numeric | `n:2000` |
| `b:` | Boolean | `b:true` |
| `i:` | Text input | `i:search term` |
| `dp:` | Date preset | `dp:last_7_days` |
| `ds:` | Date (single) | `ds:2024-01-01` |
| `dr:` | Date range | `dr:2024-01-01,2024-12-31` |
### Examples
**Single filter:**
```html theme={null}
```
**Multiple filters (AND logic):**
```html theme={null}
```
**Date range filter:**
```html theme={null}
```
### Behavior
* Filters appear as pills in the omnibar — users can see what's active
* Multiple filters combine with AND logic
* Users can remove or modify filters using the pill controls
* Non-existent columns are silently ignored (no error)
* Works with both JWT-authenticated and public (anonymous) access
### Dynamic Filters from JavaScript
```html theme={null}
```
***
## Controlled Filters (`filter_` prefix)
Controlled filters are configured in the Upsolve Hub with a **control key**. They're designed for server-side or host-app control — typically hidden from end users.
For setup instructions on configuring controlled filters in the Hub, see [Controlled Filters](/embedded-bi/filters/controlled-filters).
### URL Format
```
filter_{controlKey}={JSON-encoded value}
```
For **Select Multi** filter types, the value **must** be a JSON array, not a plain string. Plain strings cause chart load errors.
### Examples
```html theme={null}
```
### Combining Both Filter Types
You can use omnibar (`f_`) and controlled (`filter_`) filters together in the same URL:
```html theme={null}
```
The omnibar filter shows as an interactive pill; the controlled filter is applied silently (unless configured as visible).
## Troubleshooting
### Filter Not Applying
* **Check the column name**: The `f_` prefix requires the exact table and column name from your database. Misspelled names are silently ignored.
* **Check the source type**: Use `Raw` for database columns, `Output` for computed columns.
* **Check value encoding**: For `f_` filters, include the type prefix (`sm:`, `s:`, `n:`, etc.). For `filter_` controlled filters, use URL-encoded JSON.
### Controlled Filter Causes Chart Errors
* **Use JSON arrays for Select Multi**: `filter_key=%5B%22value%22%5D` (which is `["value"]` URL-encoded), not `filter_key=value`.
* **Verify the control key matches**: The `filter_` suffix must match the exact `controlKey` configured in the Hub filter settings.
### Filters Visible When They Should Be Hidden (or Vice Versa)
* Omnibar (`f_`) filters are always visible as pills
* Controlled (`filter_`) filters are hidden by default. Set `isVisible: true` in the Hub filter configuration to make them visible.
# Build and Deploy Your First Dashboard
Source: https://docs.upsolve.ai/embedded-bi/introduction
Welcome to **Upsolve AI**, your go-to platform for enabling **embedded business intelligence** for your customers. By following this example we will walk you through the essentials of connecting your data, setting up tenancy, building your first dashboard, sharing your insights, and deploying your dashboards in your product.
![Embedded Business Intelligence Dashboard By Upsolve AI (upsolve.ai)]()
![Embedded Business Intelligence Dashboard By Upsolve AI (upsolve.ai)]()
## Connecting data:
We’ve already set up a **sample database** to help you quickly explore and test the platform without needing to connect your own data source.
To view this connection, please go to the **Connect** tab as seen below:
![Upsolve Integration With SQL based Databases like Postgres, Snowflake or BigQuery]()
![Upsolve Integration With SQL based Databases like Postgres, Snowflake or BigQuery]()
If you would like to connect to a different data source, please have a look at how to [set up your connection](data-connection/data-connection).
## Build your first dashboard
In order to build your first dashboard you should begin by creating your first chart before adding to the dashboard.
### Build your first chart
It requires five steps to build your first chart. In the below example, you will build a grouped bar chart using the aggregations table.
To begin click on **+ New Chart** to open the interface.
Search and select the table(s) that contain the relevant data for the chart you want to create, in this case, aggregations table.
![Creating Chart in Upsolve AI (upsolve.ai)]()
![Creating Chart in Upsolve AI (upsolve.ai)]()
The list of available raw table(s) is generated after the [data connection](/embedded-bi/data-connection/data-connection) is set up.
Write any SQL query to transform the table(s) you selected to a single table that you will consume in Step 3 for chart building. You could write SQL to join, filter on table(s) and more.
For this example, you simply want to use the whole **aggregations** table for chart building. Click on the **+ aggregations**, this will create `SQL SELECT * FROM "aggregations" `\
in the SQL editor.
Click on **Preview data** to run the SQL and **Next** once you see the preview data.
![Writing SQL Query To Build Chart]()
![Writing SQL Query To Build Chart]()
If you are more advanced with SQL, you can add more tables and customise your SQL.
Select the relevant chart type, and choose the corresponding x- & y-axis columns for the charting. This chart will be consumed in step 4 when you create your Chart Filters.
In this example, select a **Grouped bar**, and choose **country** as the x-axis and **demand** as the y-axis.
the corresponding x- & y-axis
For future charts see [Chart Configuration](/embedded-bi/charts/create-chart) for all chart settings.
![Configuring Chart in Upsolve AI (upsolve.ai)]()
![Configuring Chart in Upsolve AI (upsolve.ai)]()
To add chart filters you will undertake two steps, firstly deciding the table to filter on and secondly configuring the filter. In this example, we will guide you through both of these steps.
In this example we will select the **Computed Table**
**1. Selected backing tables:** The list of selected backing tables in Step 1 is surfaced here. Filter any column in any selected backing table(s) that is powering this specific chart
**2. Table:** filter any column from the table that was computed by the SQL written in Step 3.
**3. Variables in the chart SQL:** inject variables into the SQL code that is backing this chart
![Selecting Table From Database in Upsolve AI (Upsolve.ai)]()
![Selecting Table From Database in Upsolve AI (Upsolve.ai)]()
**1. Table:** select the table to filter or "Query variables", in this example select **Computed Table**
**2. Column:** select the column or variable to filter on, in this example select **Country**
**3. Filter:** configure the filter as needed. Options will be provided based on the type of filter selected. In this example chose **Select** and **Select Multi**
![Configuring Table Filters in Upsolve AI (Upsolve.ai)]()
![Configuring Table Filters in Upsolve AI (Upsolve.ai)]()
For a more comprehensive overview, check out our [in-depth filtering guide](/embedded-bi/filters/filters-in-depth).
Click **Save and add to dashboard** when you are happy with your chart, this will save the chart. You could also
create a copy of the chart by clicking **Duplicate**.
![Hero Light]()
![Hero Dark]()
### Configure your dashboard
You are able to customise your dashboard by adding dashboard filters, controlled filters and additional charts.
To resize charts you can hover over the dashboard, and then simply drag the chart to the size you would like.
To move a chart you can hover over the dashboard, and then simply drag and drop the chart.
#### Add dashboard filters
To add a dashboard filter please click on **New Filter**,
Give your filter a name, and set the type and subtype. In this case, we are going to set a Select filter with single options allowed.
![Adding Filters in Dashboard]()
![Adding Filters in Dashboard]()
Select the charts you would like to pass a dashboard filter to. The available charts must have the same chart filter type as your dashboard filter type. Click the green (+) button next to each filter you want to target:
In this example, the Example Chart has a **Select** Filter so we will use this chart.
![Selecting chart filter in Upsolve AI (Upsolve.ai)]()
![Selecting chart filter in Upsolve AI (Upsolve.ai)]()
Once you have created your filter, you will be able to add the filter to the dashboard.
You are able to test this by using example values, in this instance we will use **Malaysia**.
![Dashboard Filter Preview in Upsolve.ai]()
![Dashboard Filter Preview in Upsolve.ai]()
For more information on dashboard filters, check out our [Dashboard Filters
Guide](/embedded-bi/filters/filters-in-depth).
#### Add charts
To add additional charts you can go to through the steps of creating a new chart.
Alternatively if you have already built a chart you can click on **Add Chart** to use on your dashboard.
![Adding chart in dashboard]()
![Adding chart in dashboard]()
## Share your dashboard and collaborate
You can share your dashboard via email **with anyone** by clicking on the share button and sharing it, if you would like a preview of the dashboard please click on the preview.
By sharing your dashboard, they will become a team member of your organization and can view, edit and collaborate on the dashboard(s) with you.
![Sharing Dashboard and Collaborating With Team]()
![Sharing Dashboard and Collaborating With Team]()
If the invitee aceepts the invite, they will gain access to ALL dashboards and
data in your organization!
## User Authorization (Tenant Registration)
To set up **tenant registration** please follow the steps outlined in [Backend Setup](deploy-dashboards/backend-setup).
Alternatively if you are using our **sample database** and do **not intend to go live** with the dashboard this step is optional.
### Preview dashboard as your designated end users
If you have completed the [Backend Setup](deploy-dashboards/backend-setup), you are able to view the dashboard as one of your customers. Please click on the **View As** button in your hub.
If you do not intend to go live with the dashboard, and thus did not setup your backend accordingly, you could use the platform to create a testing tenant instead and preview the dashboard as the testing tenant. We have provided two sample testing tenants for you for the sample database, which you could use to preview the example dashboard built as the tenant.
![Hero Light]()
![Hero Dark]()
If you are using the **sample database** and want to add more to add more **Preview Tenants**, please click on **Home**, then **Deploy** and then **Registered Tenants**
Click on **Create tenant**
You can use the following values to simulate creating a tenant:
* **Tenant Display Name**: `{customer-1}`
* **Filter Column**: `{CustomerID}`
* **Filter Value**: `{User_1}`
![Hero Light]()
![Hero Dark]()
Any created testing tenant would not be available in live environment. They are
only for testing purpose within Upsolve AI's purpose.
## Deploy your dashboard
Please click on the Deploy button in the Hub.
![Embed and deploy dashboard as iFrame]()
![Embed and deploy dashboard as iFrame]()
You will need to get the JWT for the current user and the dashboard ID to render the correct dashboard. The JWT is generated by following the steps in [Backend Setup](deploy-dashboards/backend-setup). The dashboard ID could be found in the URL when you are viewing the dashboard you want to embed.
Below is an example of deploying component with JWT using [**Clerk middleware**](https://clerk.com/docs/references/nextjs/auth-middleware#use-after-auth-for-fine-grained-control):
```javascript theme={null}
"use client";
import React from "react";
import { useUser } from "@clerk/nextjs";
const Analytics: React.FC = () => {
// Get the user information from Clerk and extract the Upsolve token from the metadata
const { user } = useUser();
const upsolveToken = user?.publicMetadata?.["upsolveToken"];
// Embed the component into your app
return (
<>
{upsolveToken != null && typeof upsolveToken === "string" && (
)}
);
};
export default Analytics;
```
# Schedule Reports
Source: https://docs.upsolve.ai/embedded-bi/scheduled-reporting/schedule-reports
Automatically email static dashboard links to recipients on a cadence
With SMTP verified, you can schedule recurring report emails that include secure links to static dashboard snapshots. Reports arrive on time, every time, with numbers that don’t change.
### What Gets Sent
* Email subject and message body you configure
* A secure link to a static dashboard snapshot (generated at send time)
* Optional custom message for context
![schedule-reports-overview]()
### Workflow: Create a Schedule
1. Open a dashboard and click "Schedule report"
2. Name your schedule (e.g., "Weekly Executive Summary")
3. Select recipients (add multiple emails, names optional)
4. Choose the send time and timezone
5. Set the cadence via cron expression or the visual picker (e.g., every Monday 8am)
6. Customize subject and message body
7. Save and activate the schedule
### Filter and Tenant Persistence
When you create a scheduled report, any dashboard filters or tenant selection active at the time will be preserved and applied to each static snapshot generated by the schedule:
* **Dashboard Filters**: All filter values set when creating the schedule will be locked into the schedule. Each time the report is sent, the static snapshot will reflect those filter values.
* **Relative Date Filters**: Date filters using relative ranges (such as "Last 7 Days", "Month to Date", or "Last 30 Days") are treated specially. These filters will be recalculated at send time, ensuring the report always shows the most recent data for the specified period. For example, a schedule with "Last 7 Days" will always show the previous 7 days from when the email is sent, not the 7 days from when the schedule was created.
* **Static Date Filters**: Date filters with specific date ranges (such as "January 1, 2024 - January 31, 2024") remain fixed and will always show data for those exact dates.
* **Tenant Selection**: If you have selected a specific tenant using the "View as" feature when creating the schedule, the schedule will be scoped to that tenant's data permissions. Each generated snapshot will respect the tenant's data access rules.
This ensures that each report is generated with consistent filtering and data access, making it ideal for sending tenant-specific reports or filtered views to different stakeholder groups.
### Custom URL Base (Advanced)
When embedding dashboards into your product via iframe, you can customize the base URL used in email report links. This allows you to use your own domain or staging environment instead of the default `hub.upsolve.ai/share/static` URL.
**Use Cases:**
* Embedding dashboards into your product's iframe
* Using a custom domain for report links
* Pointing to a staging environment for testing
**How to Configure:**
1. When creating or editing a schedule, expand the "Advanced Options" section
2. Enter your custom URL base in the "Custom URL Base" field
* You can include the protocol (`https://` or `https://`) or omit it (defaults to `https://`)
* Examples:
* `https://{your-domain}.com/share`
* `yourdomain.com/dashboards`
3. The system will automatically append the access token to your custom URL base
**Note:** The custom URL base applies to both scheduled sends and "Send Now" test emails, ensuring consistency across all email reports.
**Example:**
If you set the custom URL base to `https://{your-domain}.com/share`, the email will contain a link like:
```
https://{your-domain}.com/share/{accessToken}
```
Instead of the default:
```
https://hub.upsolve.ai/share/static/{accessToken}
```
![custom-url-base-configuration]()
> **Note:** Only configure a custom URL base if you plan to iframe share dashboards into your product. For standard use cases, the default URL works perfectly.
Entry point:
![schedule-reports-entry]()
Modal:
![create-schedule-modal]()
Cron picker:
![cron-picker]()
Recipients:
![schedule-recipients]()
### Managing Schedules
* Toggle active/inactive
* Edit cadence, recipients, or content
* View send history and delivery logs
![schedules-list]()
![email-logs]()
### Best Practices
* Keep one schedule per audience to tailor subject/body
* Prefer static snapshot links for auditability and consistency
* Use organization email domain for higher deliverability (via SMTP verification)
# SMTP Configuration
Source: https://docs.upsolve.ai/embedded-bi/scheduled-reporting/smtp-configuration
Verify domain ownership and configure SMTP to deliver scheduled reports
Configure SMTP to send scheduled report emails from your own domain. This ensures high deliverability and a trusted sender identity.
### Prerequisites
* Access to your DNS provider (to add TXT/CNAME records)
* An email domain you control (e.g., example.com)
### Workflow: Verify Domain Ownership
1. Navigate to Settings → Email (SMTP) in the Hub
2. Enter your sending domain (e.g., reports.example.com or example.com)
3. Copy the DNS records (TXT/CNAME) displayed for verification
4. Add these records in your DNS provider
5. Click "Verify" in the Hub once DNS propagates
![smtp-domain-setup]()
![smtp-domain-setup]()
![smtp-dns-records]()
### Troubleshooting
* Verification pending: DNS can take time to propagate; check with `dig` or your DNS console
* DMARC alignment: ensure SPF/DKIM pass for better deliverability
# Static Dashboards
Source: https://docs.upsolve.ai/embedded-bi/scheduled-reporting/static-dashboards
Create and share immutable, time-stamped dashboard snapshots via secure links
Static Dashboards let you generate immutable, time-stamped snapshots of any dashboard for distribution. These snapshots are perfect for audits, board reports, monthly close, or any case where recipients should see a fixed view that won’t change as data updates.
### What is a Static Dashboard?
* A static dashboard is a frozen snapshot of a dashboard at a specific point in time
* It’s accessible via a secure, shareable link
* It does not update when underlying data changes
* Great for regulatory/compliance reporting, one-off reviews, or archiving
![static-dashboard-overview]()
### Workflow: Create and Share a Static Dashboard Link
1. Open any dashboard in the Hub
2. Click the "Share" or "Create static snapshot" action in the header
3. Confirm included filters and time range
4. Generate the static snapshot
5. Copy the secure link and share it with stakeholders
### Filter and Tenant Persistence
Static dashboard snapshots capture and preserve the exact state of the dashboard at the time of creation:
* **Dashboard Filters**: Any filters applied to the dashboard when you create the snapshot will be locked into the static view. The snapshot will always show data filtered according to those values, regardless of future changes to the dashboard.
* For one-time snapshots created via the "Create static snapshot" button, all date filters (whether relative or static) are resolved to their absolute values at the moment of creation.
* For scheduled reports, relative date filters (such as "Last 7 Days") are recalculated each time the schedule runs. See the Schedule Reports documentation for more details.
* **Tenant Selection**: If you have selected a specific tenant using the "View as" feature when creating the snapshot, the static dashboard will be scoped to that tenant's data permissions. This ensures the snapshot only includes data that tenant is authorized to see.
This makes static snapshots ideal for creating tenant-specific reports, filtered views for different audiences, or compliance documentation that requires fixed data scoping.
![create-static-snapshot]()
![static-snapshot-generated]()
![static-snapshot-generated]()
### Best Practices
* Use for board packs, MBR/QBR, audits, and month-end close
* Include the snapshot timestamp in your email or document
* Use static snapshots when downstream readers shouldn’t need to log in or to see changing numbers
# Dashboard Link Sharing
Source: https://docs.upsolve.ai/embedded-bi/sharing-and-export/dashboard-link-sharing
## Share via link
You can share your dashboard via email **with anyone** by clicking on the share button and sharing it, if you would like a preview of the dashboard please click on the preview.
By sharing your dashboard, they will become a team member of your organization and can view, edit and collaborate on the dashboard(s) with you.
![Hero Light]()
![Hero Dark]()
If the invitee aceepts the invite, they will gain access to ALL dashboards and data in your organization!
## Export as PDF 🔒
You are able to export a dashboard as a PDF by clicking on the download button as seen below.
![PDF Export Example]()
![PDF Export Example Dark]()
# Live Filter Sync & Auto-Height (iframe postMessage)
Source: https://docs.upsolve.ai/embedded-bi/sharing-and-export/iframe-messaging
Two-way sync of dashboard filter state — and content-height reporting — between an embedded Upsolve iframe and your host app
## Overview
When you embed a shared dashboard (`/share/application/:id`) as an iframe, Upsolve
keeps its URL filter state in sync with your host application over the browser
[`postMessage`](https://developer.mozilla.org/en-US/docs/Web/API/Window/postMessage)
API. This lets you:
* **Read** the embedded view's filter state — e.g. mirror it into your own URL so
the dashboard's filtered state is deep-linkable and survives a refresh.
* **Drive** the embedded view from your own UI — push filter changes down without
reloading the iframe.
* **Size the iframe to its content** so the dashboard scrolls with your page
instead of inside a fixed-height box — see [Auto-height](#auto-height).
The iframe emits a message on **every** filter change (the filter omnibar updates
the URL via the History API, and we emit on those updates too — not just full
navigations), accepts filter params you push down, and announces when it is ready
to receive them.
This applies to embeds of `/share/application/:id`. Filters are expressed as
URL query params — see [URL Filter Parameters](/embedded-bi/filters/url-filter-parameters)
for the `f_` omnibar param format.
## Message schema
Every message has a `type` and a numeric version `v`. Pin to the version you build
against; we bump `v` for any breaking change to a payload shape.
| Direction | `type` | Payload | Meaning |
| --------------- | ------------------ | ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| iframe → parent | `embed:ready` | `{ v }` | The iframe's listener is live. Safe to send the initial params. |
| iframe → parent | `embed:params` | `{ v, search }` | The iframe's filter state changed. `search` is a query string (leading `?`) containing **only filter params** (`f_*` / `filter_*`) — auth tokens and other internal params are intentionally stripped (see Security). Empty string means no active filters. |
| iframe → parent | `embed:height` | `{ v, height }` | The iframe's content height in CSS pixels. Only sent when you opt in with `?autoHeight=true`; see [Auto-height](#auto-height). |
| parent → iframe | `embed:set-params` | `{ v?, search }` | Apply these params to the embedded view. `search` may include or omit the leading `?`; an empty string clears all filters. Messages with a `v` newer than the iframe understands are ignored. |
Current version: **`v: 1`**.
## Security: lock down origins
The iframe never posts to `"*"`. Instead, **you declare your origin** to the iframe
via a `parentOrigin` query param in the iframe `src`:
```
https://ai-hub.upsolve.ai/share/application/{appId}?jwt={token}&parentOrigin=https://your-app.com
```
The iframe validates `parentOrigin` is a well-formed `http(s)` origin, only accepts
messages whose `event.origin` matches it, and only posts back to it. On your side,
**always** validate `event.origin` against the iframe's origin before trusting a
message, and pass an explicit `targetOrigin` when you post — never `"*"`.
`embed:params` only ever carries **filter params** (`f_*` / `filter_*`). The
iframe deliberately strips auth tokens (`jwt`, `supabaseToken`, `dbAuthToken`) and
internal params (`parentOrigin`, `transparent`) before posting, so mirroring
`embed:params.search` into your own URL bar will never expose an Upsolve token in
your address bar, browser history, or `Referer` headers. Keep those tokens in the
iframe `src` you construct — they don't need to round-trip through `postMessage`.
## Auto-height
An iframe has no intrinsic height. It is a replaced element with a spec default
of 300×150 and no content-driven sizing, so `height: auto` does nothing and
`height: 100%` only resolves if every ancestor has a definite height. Whatever
height you pick, the dashboard lays itself out inside it and everything past it
scrolls **inside** the frame: your reader gets a second scrollbar, your page can
never scroll the dashboard as part of its own content, and a print or
full-page screenshot of your page captures only the visible slice.
Add `autoHeight=true` to the iframe `src` and the embed instead:
1. lays out from its content rather than pinning itself to the iframe viewport
(nothing inside the embed scrolls on its own), and
2. posts `embed:height` with that content height, on load and on every change —
a chart finishing its load, a filter bar wrapping to a second line, a
re-layout after you resize the frame.
```
https://ai-hub.upsolve.ai/share/application/{appId}?jwt={token}
&parentOrigin=https://your-app.com
&autoHeight=true
```
Apply what you receive as the iframe's height. That is the whole host side:
```js theme={null}
const iframe = document.getElementById("upsolve-embed");
window.addEventListener("message", (e) => {
if (e.origin !== IFRAME_ORIGIN) return;
// Multiple embeds on one page: make sure this is the one you are sizing.
if (e.source !== iframe.contentWindow) return;
if (e.data?.type !== "embed:height") return;
iframe.style.height = `${e.data.height}px`;
});
```
`autoHeight` needs `parentOrigin` — as with every other message, the embed will
not post to `"*"`, so without a declared origin there is nothing to listen for.
### Notes
* **Give the frame a starting height anyway.** The first `embed:height` arrives
after the page mounts; until then the frame is whatever CSS says it is. A
sensible initial height (or a skeleton) avoids a visible jump from 150px.
* **Nothing scrolls inside the frame.** Auto-height removes the embed's internal
scroll containers on purpose — with one in place, content that sits outside the
normal flow ends up in its scroll area rather than in the height we report to
you, i.e. behind a scrollbar you cannot see. A dashboard wider than the frame
therefore scrolls the frame's own document horizontally rather than a panel
inside it; keep the iframe at `width: 100%` and that does not arise.
* **Sticky elements stop being sticky.** The dashboard's filter bar sticks to
the top of the *scrollport*, and in an auto-height embed there is no scrollport
inside the frame — your page owns the scrolling. The bar scrolls away with the
content.
* **Width still comes from you.** Auto-height is height only: keep the iframe at
`width: 100%` and the dashboard reflows to it.
* **Heights are capped at 20,000px.** A page that somehow sized itself off the
viewport could otherwise grow a little on every round trip; past the cap the
embed simply scrolls internally again.
* **Older embeds ignore the param.** If you host a pinned Upsolve version that
predates this, `autoHeight=true` is inert and no `embed:height` ever arrives —
so keep whatever height fallback you have today and let a reported height
supersede it.
## Consumer reference implementation
Drop this into the page that hosts the iframe. It mirrors the iframe's filter state
into your own URL bar, and (optionally) pushes your URL's filters down on load.
```js theme={null}
const IFRAME_ORIGIN = "https://ai-hub.upsolve.ai";
const iframe = document.getElementById("upsolve-embed");
// Build the iframe URL: forward your filters + declare your origin.
iframe.src =
`${IFRAME_ORIGIN}/share/application/${appId}?jwt=${token}` +
`${location.search ? "&" + location.search.slice(1) : ""}` +
`&parentOrigin=${encodeURIComponent(location.origin)}`;
window.addEventListener("message", (e) => {
// 4. Origin lockdown — only trust the embedded iframe.
if (e.origin !== IFRAME_ORIGIN) return;
// 3. Handshake — push the initial params once the iframe is listening.
if (e.data?.type === "embed:ready") {
iframe.contentWindow.postMessage(
{ type: "embed:set-params", v: 1, search: location.search },
IFRAME_ORIGIN
);
}
// 1. The iframe's filters changed — mirror them into your URL bar.
if (e.data?.type === "embed:params") {
const url = new URL(location.href);
url.search = e.data.search;
history.replaceState(null, "", url); // mirror up without spamming back button
}
});
```
### Pushing filters live
To change the embedded view's filters from your own UI after load — without
reloading the iframe — post an `embed:set-params` message at any time:
```js theme={null}
function setEmbedFilter(search) {
iframe.contentWindow.postMessage(
{ type: "embed:set-params", v: 1, search },
IFRAME_ORIGIN
);
}
// e.g. filter to a single discipline
setEmbedFilter("f_Raw.olympics_summer.discipline=sm:Athletics");
// e.g. clear all filters
setEmbedFilter("");
```
The dashboard re-renders against the new filters in place.
## Notes & gotchas
* **Send params only after `embed:ready`.** If you post `embed:set-params` before
the iframe is listening, it is silently dropped. Wait for the handshake.
* **`embed:params` echoes.** After you apply `embed:set-params`, the iframe will
emit an `embed:params` reflecting the canonical (normalised) query string. The
reference handler above mirrors it with `history.replaceState`, which does not
re-emit — so there is no feedback loop.
* **`search` is the source of truth.** Both directions speak the URL query string,
so the same `f_`/`filter_` param formats documented in
[URL Filter Parameters](/embedded-bi/filters/url-filter-parameters) apply.
* **Auth tokens stay in the iframe `src`.** Keep `jwt` / `dbAuthToken` in the
iframe URL you construct; they don't need to be (and shouldn't be) round-tripped
through `postMessage`.
# ActiveCampaign
Source: https://docs.upsolve.ai/integrations/activecampaign
Discover a seamless integration process! Learn how to connect ActiveCampaign to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting ActiveCampaign to Upsolve, allowing you to seamlessly access your ActiveCampaign data within the Upsolve platform.
## Prerequisites
Before you start, you will need an API Key and an API URL from ActiveCampaign.
Follow the steps below to obtain it:
1. Visit the [ActiveCampaign website](https://www.activecampaign.com/) and log in.
2. Click on the "Settings" button and click on the "Developer" button on the side bar, as shown in the image below:
![]()
3. Copy the API key and the API URL.
![]()
## Connecting ActiveCampaign to Upsolve
Now that you have your ActiveCampaign API URL and API Key, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "ActiveCampaign" as your data source.
4. Enter a name for your data source name and paste the ActiveCampaign API URL and API Key into the input fields.
![]()
5. Click "Authorize" to create your ActiveCampaign data source.
Congratulations! You've successfully connected ActiveCampaign to Upsolve. You can now access and manage your ActiveCampaign data seamlessly within the Upsolve platform.
# Airtable
Source: https://docs.upsolve.ai/integrations/airtable
Discover a seamless integration process! Learn how to connect Airtable to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Airtable to Upsolve, allowing you to seamlessly access your Airtable data within the Upsolve platform.
## Prerequisites
Before you start, you will need an Personal Access Token from Airtable.
Follow the steps below to obtain it:
1. Visit the [Airtable website](https://airtable.com).
2. Click on the "Developer Hub" button, as shown in the image below:
![]()
3. Create an API Token by clicking the "Create Token" button. You can select the necessary scopes and workspaces for your integration:
![]()
4. Once the token is created, it will be displayed in a modal. Copy the API Token for later use:
![]()
## Connecting Airtable to Upsolve
Now that you have your Airtable API Token, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Airtable" as your data source.
4. Enter a name for your data source name and paste the Airtable API Token into the "Personal Access Token" input field.
![]()
5. Click "Authorize" to create your Airtable data source.
Congratulations! You've successfully connected Airtable to Upsolve. You can now access and manage your Airtable data seamlessly within the Upsolve platform.
# Amazon Athena
Source: https://docs.upsolve.ai/integrations/amazon-athena
Learn how to connect Amazon Athena to Upsolve to query your S3 data lake directly from your dashboards.
This article guides you through the process of connecting Amazon Athena to Upsolve, allowing you to query your S3 data lake within the Upsolve platform.
Athena is serverless — there is no cluster or host to provision. Instead of a network endpoint and database login, you connect with an AWS IAM key pair and tell Athena where to stage its results in S3.
## Prerequisites
Before you start, you will need the following from AWS:
* **Region** — where your Athena workgroup and Glue catalog live (e.g. `us-east-1`)
* **Access Key ID** and **Secret Access Key** — for an IAM user or service account
* **S3 output location** — an S3 URI Athena writes query results to (e.g. `s3://my-bucket/athena-results/`)
* **Database** — the database unqualified table names resolve against at query time
* **Schemas** (optional but recommended) — comma-separated schemas to introspect, with `*` wildcards, e.g. `dw,domain_*_mart*`. Empty falls back to the Database field, then to every schema in the catalog.
* **Data catalog** (optional, defaults to `AwsDataCatalog`)
* **Workgroup** (optional, defaults to `primary`)
Athena has no username or password. The IAM key pair is the equivalent — the
official Athena JDBC driver maps the Access Key ID to `User` and the Secret
Access Key to `Password`, and Upsolve follows the same convention.
Follow the steps below to obtain them:
1. Go to the AWS Management Console and open **Athena**. Confirm the region shown in the top-right corner — this is the region you will enter in Upsolve.
2. Under **Query editor → Settings**, check the **Location of query result**. If it is blank, set it to an S3 prefix you control. This is your S3 output location.
3. In **Data sources**, note the data catalog (usually `AwsDataCatalog`) and the database containing your tables. If your tables are not yet catalogued, run an AWS Glue crawler over your S3 data first.
4. Open **IAM** and create (or reuse) a user for Upsolve, then create an access key for it. Save the secret when it is shown — AWS will not display it again.
### IAM permissions
Attach a policy granting the Upsolve key pair at least:
* `athena:StartQueryExecution`, `athena:GetQueryExecution`, `athena:GetQueryResults`, `athena:StopQueryExecution`, `athena:GetWorkGroup`
* `glue:GetDatabase`, `glue:GetDatabases`, `glue:GetTable`, `glue:GetTables`, `glue:GetPartition`, `glue:GetPartitions` — for the databases you expose
* `s3:GetObject`, `s3:PutObject`, `s3:AbortMultipartUpload`, `s3:ListBucket`, `s3:GetBucketLocation` on the output location
* `s3:GetObject`, `s3:ListBucket`, `s3:GetBucketLocation` on the buckets your tables read from
Without write access to the output location, every query fails: Athena stages each result set in S3 before it can be read back.
`s3:ListBucket` and `s3:GetBucketLocation` are bucket-level actions and cannot
be scoped to a sub-directory by resource ARN — granting them on
`arn:aws:s3:::bucket/prefix/*` silently never matches. Use the `s3:prefix`
condition key, or give Athena a dedicated results bucket. An error mentioning
"an explicit deny in a resource-based policy" means the bucket policy is
denying the call, and no identity-policy grant can override it.
If the connection saves but the data model is empty, the Athena and results-bucket permissions are working and Glue or source-bucket access is not — the connection test runs `SELECT 1+1`, which reads no table.
## Connecting Amazon Athena to Upsolve
Now that you have your requirements, you can connect it to Upsolve using the following steps:
1. Navigate to the connections page in Upsolve.
2. Click to add a new connection and select **Athena** as your data source type.
3. Enter a name for your connection, then fill in the details:
* Region
* Access Key ID
* Secret Access Key
* S3 output location
* Database
* Data catalog (optional)
* Workgroup (optional)
4. Click "Test & Create Connection" to validate the credentials and create your Athena connection.
Congratulations! You've successfully created a new data source between Amazon Athena and Upsolve.
## Things to know about Athena
* **Athena uses Trino/Presto SQL.** Identifiers are double-quoted, and date handling uses `date_trunc` and `date_format` rather than their Postgres or MySQL equivalents.
* **Queries are billed per byte scanned.** Upsolve therefore disables automatic selectable-column detection for Athena connections — detecting it would run a `COUNT` per table on every metadata refresh. Partitioning your tables and filtering on the partition keys is the most effective way to control cost.
* **Executions are asynchronous.** Upsolve polls until a query completes and gives up after 120 seconds.
* **Scope your schemas.** Athena's `information_schema` spans the whole catalog. On a catalog with thousands of schemas, introspecting all of them is slow — AWS recommends against it — so set the Schemas field to just the ones you need.
The specific steps for connecting Amazon Athena may vary based on your unique
setup. If you encounter any difficulties or have questions, please don't
hesitate to contact our Support Team for assistance.
# Amazon RDS
Source: https://docs.upsolve.ai/integrations/amazon-rds
Discover a seamless integration process! Learn how to connect Amazon RDS to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Amazon RDS to Upsolve, allowing you to seamlessly access your Amazon RDS data within the Upsolve platform.
## Prerequisites
Before you start, you will need to obtain certain values, which you can get from Amazon RDS.
Upsolve now supports three different databases: MySQL, PostgreSQL, and MariaDB. You can select any of these options. If you switch to "Aurora," your database will remain on Amazon instances. This option is only available for MySQL and PostgreSQL databases.
![]()
You need to select your AWS region and enter the URL and port. For a PostgreSQL database, you also need to enter the database name. You can obtain these values from your Amazon account.
You can authenticate using one of three methods: IAM, Secret Manager, or Password. The requirements are as follows:
**IAM**: User Name, AWS Access Key, and AWS Secret Key. Check this [article](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/database-authentication.html) to get credentials.
**Secret Manager**: Secret Manager ARN, AWS Access Key, and AWS Secret Key. Check this [article](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-secrets-manager.html) to get credentials.
**Password**: User Name and Password. Check this [article](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/database-authentication.html) to get credentials.
## Connecting Amazon RDS to Upsolve
Now that you have Amazon RDS, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Amazon RDS" as your data source.
4. Enter the values you obtained earlier.
![]()
5. Click "Connect" to create your Amazon RDS data source.
Congratulations! You've successfully connected Amazon RDS to Upsolve. You can now access and manage your Amazon RDS data seamlessly within the Upsolve platform.
# Amazon Redshift
Source: https://docs.upsolve.ai/integrations/amazon-redshift
Discover a seamless integration process! Learn how to connect Amazon Redshift to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Amazon Redshift to Upsolve, allowing you to seamlessly access your Amazon Redshift data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Amazon Redshift.
* Port
* Host
* User name
* Password
* Database name
Follow the steps below to obtain it:
1. Go to Amazon Redshift.
2. After accessing Amazon Redshift, create your cluster.
3. After clicking the "Create Cluster" button, you will be redirected to provide some information. Your password and username should match the details shown in the image below.
4. If you choose to customize the Additional Configurations settings, you have the flexibility to modify your database name. However, if you prefer to stick with the default settings, your database name will be set to "dev" and the port will remain at the default value of 5439. Once you've finished inputting your desired configurations, simply click the "Create Cluster" button located at the bottom of the page. Please be patient as the creation process may take a few minutes to complete.
5. Accessing your host's knowledge can be done through the "Properties" tab.
## Connecting Amazon Redshift to Upsolve
Now that you have your requirements, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Amazon Redshift" as your data source type.
4. Enter a name for your data source name and data source details below:
* User name
* Password
* Port
* Host
* You can enable the "Connect through TLS" option if SSL is active.
![Screenshot 2025-05-08 at 09.57.42.png]()
5. Click "Connect" to create your Amazon Redshift data source.
Congratulations! You've successfully created a new data source between Amazon Redshift and Upsolve. You can now access and manage your Amazon Redshift data seamlessly within the Upsolve platform.
The specific steps for connecting Amazon Redshift may vary based on your
unique setup. If you encounter any difficulties or have questions, please
don't hesitate to contact our Support Team for assistance.
# BigQuery
Source: https://docs.upsolve.ai/integrations/big-query
Discover a seamless integration process! Learn how to connect BigQuery to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting BigQuery to Upsolve, allowing you to seamlessly access your BigQuery data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from BigQuery.
* Project ID
* Parent Project ID (optional)
* Credentials in JSON format
Follow the steps below to obtain it:
1. Go to the BigQuery dashboard. You can find your Project ID in the blue squared area.
![]()
2. Visit the [Credentials page](https://console.cloud.google.com/apis/credentials).
3. Select "+Create credentials" and then choose "Service account."
![]()
4. Enter a name for the service account (e.g., "Upsolve BigQuery Read Account") and select "Create and Continue."
![]()
5. Ensure the service account has the following permissions:
* BigQuery Data Editor
* BigQuery User
![]()
6. You can skip the optional user access step and select "Done."
7. Your servise account will be created and click on the service account email.
![]()
8. Click the "Add Key" button and select "Create New Key."
![]()
9. Choose the "JSON" file format and click the "Create" button. This will download a JSON key file that you'll need for the integration.
![]()
## Connecting BigQuery to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect BigQuery to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "BigQuery" as your data source.
4. Enter a name for your data source and upload the credentials JSON file you downloaded in step 9.
![]()
5. Click "Next" to create your BigQuery data source.
Congratulations! You've successfully connected BigQuery to Upsolve. You can now access and manage your BigQuery data seamlessly within the Upsolve platform.
# Clickhouse
Source: https://docs.upsolve.ai/integrations/clickhouse
Discover a seamless integration process! Learn how to connect Clickhouse to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Clickhouse to Upsolve, allowing you to seamlessly access your Clickhouse data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Clickhouse.
* URL
* PORT
* User Name
* Password
* SSL (Toggle "Connect Through TLS" switch)
Follow the steps below to obtain it:
1. Open the Clickhouse dashboard.
2. Click on the "Connect" button as shown in the image below.
![]()
3. It will open a modal and show the data source detail such as URL, PORT, User Name, and Password.
![]()
## Connecting Clickhouse to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Clickhouse to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Clickhouse" as your data source.
4. Enter a name for your data source, input the credentials you obtained in the previous steps, and toggle the "Connect Through TLS" switch if SSL is required.
![]()
5. Click "Next" to create your Clickhouse data source.
Congratulations! You've successfully connected Clickhouse to Upsolve. You can now access and manage your Clickhouse data seamlessly within the Upsolve platform.
# Databricks
Source: https://docs.upsolve.ai/integrations/databricks
Discover a seamless integration process! Learn how to connect Databricks to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Databricks to Upsolve, allowing you to seamlessly access your Databricks data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Databricks.
* URL
* PORT
* HTTP Path
* User
* Password
Follow the steps below to obtain it:
1. To obtain get URL, PORT and HTTP path, check this [article](https://docs.databricks.com/en/integrations/compute-details.html#:~:text=do%20the%20following%3A-,Log%20in%20to%20your%20Databricks%20workspace.,%2C%20Port%2C%20and%20HTTP%20path.).
2. To obtain get User and Password, check this [article](https://docs.databricks.com/api/account/accountusers/get).
## Connecting Databricks to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Databricks to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Databricks" as your data source.
4. Enter a name for your data source, input the credentials you obtained in the previous steps.
![]()
5. Click "Next" to create your Databricks data source.
Congratulations! You've successfully connected Databricks to Upsolve. You can now access and manage your Databricks data seamlessly within the Upsolve platform.
# Dynamics 365
Source: https://docs.upsolve.ai/integrations/dynamics-365
Discover a seamless integration process! Learn how to connect Dynamics 365 to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Dynamics 365 to Upsolve, allowing you to seamlessly access your Dynamics 365 data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Dynamics 365.
* Dynamics 365 URL
* Client ID (required for advanced authentication)
* Client Secret (required for advanced authentication)
Follow the steps below to obtain it:
1. To obtain the Dynamics 365 URL, refer to this [article](https://community.dynamics.com/blogs/post/?postid=d1ebf5d8-0a0c-41af-91ff-cd25f1cb37f7).
2. To obtain the Client ID and Client Secret, refer to this [article](https://learn.microsoft.com/en-us/dynamics365/finance/localizations/global/universal-tax-rate-api-how-to-setup-clientid-and-clientsecret).
## Connecting Dynamics 365 to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Dynamics 365 to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Dynamics 365" as your data source.
4. Enter a name for your data source, input the credentials you obtained in the previous steps.
![]()
5. Click "Sign in" to create your Dynamics 365 data source.
Congratulations! You've successfully connected Dynamics 365 to Upsolve. You can now access and manage your Dynamics 365 data seamlessly within the Upsolve platform.
# ElasticSearch
Source: https://docs.upsolve.ai/integrations/elastic-search
Discover a seamless integration process! Learn how to connect ElasticSearch to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting ElasticSearch to Upsolve, allowing you to seamlessly access your ElasticSearch data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from ElasticSearch.
* Host
* Port
* User name
* SSL (Enable this option if you want to use SSL)
Follow the steps below to obtain it:
1. To obtain the Host and Port, navigate to the developer page and copy the ElasticSearch cluster URL. You can extract the Host and Port from the URL.
![]()
2. Navigate to the developer page and click on the "Reset Password" button as shown in the image.
![]()
3. To obtain User name and Password, cLick on the Reset Password button as in the image, and your default user name is "elastic".
![]()
## Connecting ElasticSearch to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect ElasticSearch to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "ElasticSearch" as your data source.
4. Enter a name for your data source, input the credentials you obtained in the previous steps.
![]()
5. Click "Authorize" to create your ElasticSearch data source.
Congratulations! You've successfully connected ElasticSearch to Upsolve. You can now access and manage your ElasticSearch data seamlessly within the Upsolve platform.
# Excel 365
Source: https://docs.upsolve.ai/integrations/excel-365
Discover a seamless integration process! Learn how to connect Excel 365 to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Excel 365 to Upsolve, allowing you to seamlessly access your Excel 365 data within the Upsolve platform.
## Connecting Excel 365 to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Excel 365" as your data source.
4. Enter a name for your data source.
![]()
5. Click the "Sign in" button to log in with your Excel 365 account. A pop-up window will appear, redirecting you to the Excel 365 login page.
6. Click the "Accept" button which will create your Excel 365 data source.
Congratulations! You've successfully connected Excel 365 to Upsolve. You can now access and manage your Excel 365 data seamlessly within the Upsolve platform.
# Facebook Ads
Source: https://docs.upsolve.ai/integrations/facebook-ads
Discover a seamless integration process! Learn how to connect Facebook Ads to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Facebook Ads to Upsolve, allowing you to seamlessly access your Facebook Ads data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Facebook Ads.
* Default Ad Account ID
* Client ID (required for advanced authentication)
* Client Secret (required for advanced authentication)
Follow the steps below to obtain it:
1. To obtain the Default Ad Account ID, refer to this [article](https://www.facebook.com/business/help/1492627900875762).
2. To obtain the Client ID and Client Secret, refer to this [article](https://developers.facebook.com/docs/facebook-login/guides/advanced/manual-flow/#confirm).
## Connecting Facebook Ads to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Facebook Ads to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Facebook Ads" as your data source.
4. Enter a name for your data source, input the credentials you obtained in the previous steps.
![]()
5. Click "Sign in" to create your Facebook Ads data source.
Congratulations! You've successfully connected Facebook Ads to Upsolve. You can now access and manage your Facebook Ads data seamlessly within the Upsolve platform.
# Firebase
Source: https://docs.upsolve.ai/integrations/firebase
Discover a seamless integration process! Learn how to connect Firebase to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Firebase to Upsolve, allowing you to seamlessly access your Firebase data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Firebase.
* Service Account JSON
Follow the steps below to obtain it:
1. To obtain the Service Account JSON, navigate to the Firebase Console, go to the "Project Settings" page, select the "Service Accounts" tab, and click on "Generate New Private Key." This will download a JSON file containing your service account credentials.
![]()
## Connecting Firebase to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Firebase" as your data source.
4. Enter a name for your data source.
5. Upload the Service Account JSON file you downloaded in the previous step.
![]()
5. Click "Next" to create your Firebase data source.
Congratulations! You've successfully connected Firebase to Upsolve. You can now access and manage your Firebase data seamlessly within the Upsolve platform.
# Github
Source: https://docs.upsolve.ai/integrations/github
Discover a seamless integration process! Learn how to connect Github to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Github to Upsolve, allowing you to seamlessly access your Github data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Github.
* Personal Access Token
* Organization Name
Follow the steps below to obtain it:
1. To obtain get Personal Access Token, check this [article](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic).
2. To obtain get Organization Name, check this [article](https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-your-membership-in-organizations/accessing-an-organization).
## Connecting Github to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Github to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Github" as your data source.
4. Enter a name for your data source, input the credentials you obtained in the previous steps.
![]()
5. Click "Authorize" to create your Github data source.
Congratulations! You've successfully connected Github to Upsolve. You can now access and manage your Github data seamlessly within the Upsolve platform.
# Google Ads
Source: https://docs.upsolve.ai/integrations/google-ads
Discover a seamless integration process! Learn how to connect Google Ads to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Google Ads to Upsolve, allowing you to seamlessly access your Google Ads data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Google Ads.
* Client ID (required for advanced authentication)
* Client Secret (required for advanced authentication)
* Developer Token (required for advanced authentication)
* Customer ID
Follow the steps below to obtain it:
1. To obtain get Client ID and Client Secret, go to the [Goole Developers Console](https://console.cloud.google.com/apis/dashboard).
2. Click Select a project ➝ New Project ➝ the Create button.
![]()
3. Enter your Project name ➝ click the Create button.
![]()
4. Click OAuth consent screen in the left side menu ➝ choose User Type ➝ click the Create button.
![]()
5. Add Application name ➝ Support email ➝ Authorized domain ➝ Developer content information ➝ click the Save and Continue button.
6. Complete all 4 steps in OAuth consent screen ➝ click the Back to Dashboard button.
![]()
7. Go to Credentials ➝ click Create Credentials ➝ select OAuth client ID from the dropdown list.
![]()
8. Open the dropdown list Application type ➝ select Web application ➝ enter the name of your OAuth 2.0 client.
9. Enter your site URL in Authorized JavaScript origins ➝ in Authorized redirect URIs, enter the page URL where you wanted your users redirected back after they have authenticated with Google ➝ click the Create button.
![]()
10. Copy your Client ID and Client Secret.
![]()
11. To obtain the Developer Token, refer to this [article](https://developers.google.com/google-ads/api/docs/get-started/dev-token).
12. To obtain the Customer ID, refer to this [article](https://support.google.com/google-ads/answer/1704344?hl=en).
## Connecting Google Ads to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Google Ads to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Google Ads" as your data source.
4. Enter a name for your data source, input the credentials you obtained in the previous steps.
![]()
5. Click "Sign in with Google" to create your Google Ads data source.
Congratulations! You've successfully connected Google Ads to Upsolve. You can now access and manage your Google Ads data seamlessly within the Upsolve platform.
# Google Analytics
Source: https://docs.upsolve.ai/integrations/google-analytics
Discover a seamless integration process! Learn how to connect Google Analytics to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Google Analytics to Upsolve, allowing you to seamlessly access your Google Analytics data within the Upsolve platform.
Follow these steps to connect Google Analytics to Upsolve:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal, select "Google Analytics" as your data source.
4. Enter a name for your data source.
5. Click the "Signing in to Google Analytics" button. This will open a modal to allow Upsolve access to your Google Analytics account.
![]()
6. In the modal, click the "Allow" button to grant Upsolve access to your Google Analytics data.
![]()
Congratulations! You've successfully connected Google Analytics to Upsolve. You can now access and manage your Google Analytics data seamlessly within the Upsolve platform.
# Google Sheets
Source: https://docs.upsolve.ai/integrations/google-sheets
Discover a seamless integration process! Learn how to connect Google Sheets to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Google Sheets to Upsolve, allowing you to seamlessly access your Google Sheets data within the Upsolve platform.
Follow these detailed steps to establish a data source between Google Sheets and Upsolve:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal, select "Google Sheets" as your data source.
4. Enter a name for your data source.
5. Click the "Signing in to Google Sheets" button. This will open a modal to allow Upsolve access to your Google Sheets account.
![]()
6. In the modal, click the "Allow" button to grant Upsolve access to your Google Sheets data.
![]()
Congratulations! You've successfully connected Google Sheets to Upsolve. You can now access and manage your Google Sheets data seamlessly within the Upsolve platform.
# HubSpot
Source: https://docs.upsolve.ai/integrations/hubspot
Discover a seamless integration process! Learn how to connect HubSpot to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting HubSpot to Upsolve, allowing you to seamlessly access your HubSpot data within the Upsolve platform.
## Connecting HubSpot to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal, select "HubSpot" as your data source.
4. Enter a name for your data source name, click the Sign In button. This will open a modal to allow Upsolve access to your HubSpot account.
5. Allow Upsolve access to your HubSpot account.
![]()
Congratulations! You've successfully connected HubSpot to Upsolve. You can now access and manage your HubSpot data seamlessly within the Upsolve platform.
# Instagram
Source: https://docs.upsolve.ai/integrations/instagram
Discover a seamless integration process! Learn how to connect Instagram to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Instagram to Upsolve, allowing you to seamlessly access your Instagram data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Instagram.
* Client ID (required for advanced setup)
* Client Secret (required for advanced setup)
* Business Account ID
Follow the steps below to obtain it:
1. To obtain Client ID and Client Secret, check this [article](https://developers.facebook.com/docs/facebook-login/guides/access-tokens/#clienttokens).
2. To obtain Business Account ID, check this [article](https://www.facebook.com/business/help/1181250022022158?id=180505742745347).
## Connecting Instagram to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Instagram to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Instagram" as your data source.
4. Enter a name for your data source, input the credentials you obtained in the previous steps.
![]()
5. Click "Sign in" to authenticate and grant Upsolve access to your Instagram account.
Congratulations! You've successfully connected Instagram to Upsolve. You can now access and manage your Instagram data seamlessly within the Upsolve platform.
# Intercom
Source: https://docs.upsolve.ai/integrations/intercom
Discover a seamless integration process! Learn how to connect Intercom to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Intercom to Upsolve, allowing you to seamlessly access your Intercom data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Intercom.
* Client ID (required for advanced authentication)
* Client Secret (required for advanced authentication)
Follow the steps below to obtain it:
1. To obtain the Client ID and Client Secret, refer to this [article](https://developers.intercom.com/docs/build-an-integration/learn-more/authentication/setting-up-oauth#get-the-authorization-code).
## Connecting Intercom to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Intercom" as your data source.
4. Enter a name for your data source and input the Client ID and Client Secret you obtained in the previous step.
![]()
5. Click "Sign in" to create your Intercom data source and allow Upsolve access to your Intercom account.
Congratulations! You've successfully connected Intercom to Upsolve. You can now access and manage your Intercom data seamlessly within the Upsolve platform.
# Data Plane
Source: https://docs.upsolve.ai/integrations/introduction
Connect your data sources into a unified SQL Layer
## Connect your sources
The Data Plane is Upsolve's premium data connection platform. It supports 65+ connections and allows you to query across all your connected sources with SQL and AI.
![]()
# JIRA Cloud
Source: https://docs.upsolve.ai/integrations/jira
Discover a seamless integration process! Learn how to connect Jira to Upsolve effortlessly for streamlined transactions.
Experience a smooth integration process with our JIRA Cloud integration. This guide will walk you through the steps to connect JIRA Cloud with Upsolve, enabling you to effortlessly manage your JIRA data within the Upsolve platform.
## Prerequisites
Before you begin, you'll need to obtain an API Key from JIRA Cloud. Here's how to do it:
Follow the steps below to obtain it:
1. To generate your API Key, navigate to [Atlassian API Key Generation](https://id.atlassian.com/manage-profile/security/api-tokens) and follow the instructions to create your API Key.
![]()
2. To include your service URL, visit this [link](https://id.atlassian.com/manage-profile/products) and get your service URL it will be in the format of `https://<>.atlassian.net/`.
![]()
## Connecting JIRA Cloud to Upsolve
With your API Key and service URL at hand, you're ready to connect JIRA Cloud to Upsolve. Follow these steps:
1. Go to the settings page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the pop-up that appears, choose "JIRA Cloud" as your data source.
4. Input your data source name, API Key, and service URL. The username is your email address which you entered your account with.
![]()
5. Click on "Authorize" to establish your JIRA Cloud data source.
And that's it! You have successfully integrated JIRA Cloud with Upsolve. You can now access and manage your JIRA data directly within the Upsolve platform. Enjoy the seamless experience!
# LinkedIn Ads
Source: https://docs.upsolve.ai/integrations/linkedin-ads
Discover a seamless integration process! Learn how to connect LinkedIn Ads to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting LinkedIn Ads to Upsolve, allowing you to seamlessly access your LinkedIn Ads data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from LinkedIn Ads.
* Client ID (required for advanced authentication)
* Client Secret (required for advanced authentication)
* Default Account ID
Follow the steps below to obtain it:
1. To obtain the Client ID and Client Secret, go to the [LinkedIn developers site](https://www.linkedin.com/developer/apps).
2. Click Create App and complete the entire information. Specify your company name or create a new company page.
3. Click Create App. The App Settings is displayed.
4. Select the Auth tab.
5. Copy the Client ID and Client Secret.
6. To obtain the Default Account ID, [sign in to Campaign Manager.](https://www.linkedin.com/campaignmanager/).
7. The ads account name is in the upper-left corner of the page. The account ID number is below the account name.
8. To see your other ad accounts, click the account name in the upper-left corner of the page. The account ID number appears below each name.
9. For a detailed list of all your ad accounts, click the ads account name in the upper-left corner of the page, then click View all accounts at the bottom of the left menu. The account ID will appear below each account name.
10. Copy the Default Account ID.
## Connecting LinkedIn Ads to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "LinkedIn Ads" as your data source.
4. Enter a name for your data source and input credentials you obtained in the previous step.
![]()
5. Click "Sign in" to create your LinkedIn Ads data source and allow Upsolve access to your LinkedIn Ads account.
Congratulations! You've successfully connected LinkedIn Ads to Upsolve. You can now access and manage your LinkedIn Ads data seamlessly within the Upsolve platform.
# Mailchimp Marketing
Source: https://docs.upsolve.ai/integrations/mailchimp-marketing
Discover a seamless integration process! Learn how to connect Mailchimp Marketing to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Mailchimp Marketing to Upsolve, allowing you to seamlessly access your Mailchimp Marketing data within the Upsolve platform.
## Prerequisites
Follow the steps below to connect it.
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal, select "Mailchimp Marketing" as your data source.
4. Enter a name for your data source and click the "Signing in to Mailchimp Marketing" button.
![]()
5. Once clicking, it will open a modal for logging in to your Mailchimp account.
![]()
6. In the Mailchimp login modal, enter your Mailchimp credentials
7. After logging in, you will be asked to allow Upsolve access to your Mailchimp Marketing account. Click the "Allow" button to proceed.
![]()
Congratulations! You've successfully connected Mailchimp Marketing to Upsolve. You can now access and manage your Jira data seamlessly within the Upsolve platform.
# Mailchimp Transactional
Source: https://docs.upsolve.ai/integrations/mailchimp-transactional
Discover a seamless integration process! Learn how to connect Mailchimp Transactional to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Mailchimp Transactional to Upsolve, allowing you to seamlessly access your Mailchimp Transactional data within the Upsolve platform.
## Prerequisites
Before you start, you will need an API Key from Mailchimp Transactional.
Follow the steps below to obtain it:
1. Go to this [link](https://us21.admin.mailchimp.com/account/api/) to create or obtain your Mailchimp Transactional API Key.
2. Click the "Create a Key" button to generate your API Key.
![]()
## Connecting Mailchimp Transactional to Upsolve
Now that you have your requirements, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Mailchimp Transactional" as your data source.
4. Enter your data source name and API Key.
![]()
5. Click "Authorize" to create your Mailchimp Transactional data source.
Congratulations! You've successfully created a new data source between Mailchimp Transactional and Upsolve. You can now access and manage your Mailchimp Transactional data seamlessly within the Upsolve platform.
# MariaDB
Source: https://docs.upsolve.ai/integrations/mariadb
Discover a seamless integration process! Learn how to connect MariaDB to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting MariaDB to Upsolve, allowing you to seamlessly access your MariaDB data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from MariaDB.
* Host (Endpoint without the port or database name)
* Port (The port used to connect to your database)
* User (The username of the service account)
* Password (The password for the service account user)
* SSL, if active (Toggle "Connect Through TLS" switch)
## Connecting MariaDB to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "MariaDB" as your data source.
4. Fill out the data source details form as shown in the image below, including:
* Host (Endpoint without the port or database name)
* Port (The port used to connect to your database)
* User (The username of the service account)
* Password (The password for the service account user)
* Connect Through TLS (Toggle switch if SSL is active)
5. Click "Next" to establish the data source.
![]()
Congratulations! You've successfully created a new data source between MariaDB and Upsolve. You can now access and manage your MariaDB data seamlessly within the Upsolve platform.
# MongoDB
Source: https://docs.upsolve.ai/integrations/mongodb
Discover a seamless integration process! Learn how to connect MongoDB to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting MongoDB to Upsolve, allowing you to seamlessly access your MongoDB data within the Upsolve platform.
**Note**: Before you begin, ensure that Upsolve has been granted the necessary read permissions as shown in the image below.
![]()
## Prerequisites
**Important**: Ensure that the Upsolve IP address 34.42.188.247 is added to your firewall's safelist to enable access to your MongoDB data.
Before you start, you will need the following information from MongoDB.
MongoDB data source has two type connection way;
* MongoDB Atlas
* Password (The password for the MongoDB Atlas user)
* Atlas Connection String (The connection string for MongoDB Atlas)
* You can get it on MongoDB Atlas page as in the below image.
![]()
* MongoDB Manual (Self-Hosted)
* Host (Endpoint without the port or database name)
* Port (The port used to connect to your database)
* User (The username of the service account)
* Password (The password for the service account user)
* Database Name (The database name of your service)
* Multiple Replica Seed (Indicate whether MongoDB Atlas is being utilized)
## Connecting MongoDB to Upsolve
To connect MongoDB to Upsolve, follow these steps based on your MongoDB setup:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "MongoDB" as your data source.
4. Select the connection type based on your MongoDB setup.
* If you select **Atlas**;
1. Enter the connection string and password you have gathered.
![]()
* If you select **Manual**;
1. Enter the host, port, user, password, database name, and any other required details.
![]()
5. Click **Authorize** to complete the data source connection.
Congratulations! You have successfully connected MongoDB to Upsolve. You can now effortlessly access and manage your MongoDB data within the Upsolve platform.
# MSSQL
Source: https://docs.upsolve.ai/integrations/mssql
Discover a seamless integration process! Learn how to connect MSSQL to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting MSSQL to Upsolve, allowing you to seamlessly access your MSSQL data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from MSSQL.
* Host (Endpoint without the port or database name)
* Port (The port used to connect to your database)
* User (The username of the service account)
* Password (The password for the service account user)
* Database Name (The database name of your service)
* SSL, if active (Toggle "Connect Through TLS" switch)
## Connecting MSSQL to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "MSSQL" as your data source.
4. Fill out the data source details form as shown in the image below, including:
* Host (Endpoint without the port or database name)
* Port (The port used to connect to your database)
* User (The username of the service account)
* Password (The password for the service account user)
* Database Name (The database name of your service)
* Connect Through TLS (Toggle switch if SSL is active)
5. Click "Next" to establish the data source.
![]()
Congratulations! You've successfully created a new data source between MSSQL and Upsolve. You can now access and manage your MSSQL data seamlessly within the Upsolve platform.
# MySQL
Source: https://docs.upsolve.ai/integrations/mysql
Discover a seamless integration process! Learn how to connect MySQL to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting MySQL to Upsolve, allowing you to seamlessly access your MySQL data within the Upsolve platform.
Note: Users can establish a data source to their MySQL database on AWS by leveraging IAM credentials. For more details, refer to the [article on Upsolve's documentation](https://www.upsolve.com/docs/how-to-guides/how-to-connect-databases-in-aws-using-iam-credentials/).
## Prerequisites
Before you start, you will need the following information from MySQL.
* Host (Endpoint without the port or database name)
* Port (The port used to connect to your database)
* User (The username of the service account)
* Password (The password for the service account user)
* SSL, if active (Toggle "Connect Through TLS" switch)
## Connecting MySQL to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "MySQL" as your data source.
4. Fill out the data source details form as shown in the image below, including:
* Host (Endpoint without the port or database name)
* Port (The port used to connect to your database)
* User (The username of the service account)
* Password (The password for the service account user)
* Connect Through TLS (Toggle switch if SSL is active)
5. Click "Next" to establish the data source.
![]()
Congratulations! You've successfully created a new data source between MySQL and Upsolve. You can now access and manage your MySQL data seamlessly within the Upsolve platform.
# Neo4J
Source: https://docs.upsolve.ai/integrations/neo4j
Discover a seamless integration process! Learn how to connect Neo4J to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Neo4J to Upsolve, allowing you to seamlessly access your Neo4J data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Neo4J.
* Host (Endpoint without the port or database name)
* Port (The port used to connect to your database)
* User Name (The username of the service account)
* Password (The password for the service account user)
* SSL, if active (Optional)
* Bolt Schema, if active (Optional)
Follow the steps below to obtain it:
1. To obtain the User Name and Password, go to the [article](https://support.neo4j.com/s/article/6787460237843-Neo4j-Aura-Database-Authentication-Neo4j-Database-Username-Password#:~:text=When%20you%20create%20an%20instance,note%20of%20the%20username%2C%20neo4j).
2. To obtain the Host and Port, go to the [article](https://neo4j.com/docs/operations-manual/current/configuration/ports/#_ports_used_by_neo4j).
## Connecting Neo4J to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Neo4J" as your data source.
4. Enter a name for your data source and input credentials you obtained in the previous step.
5. Click "Next" to establish the data source.
![]()
Congratulations! You've successfully connected Neo4J to Upsolve. You can now access and manage your Neo4J data seamlessly within the Upsolve platform.
# Odoo
Source: https://docs.upsolve.ai/integrations/odoo
Discover a seamless integration process! Learn how to connect Odoo to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Odoo to Upsolve, allowing you to seamlessly access your Odoo data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Odoo.
* Odoo URL
* Database Name
* User Name
* Password
Follow the steps below to obtain it:
1. Odoo URL is your project URL like [https://example-app.odoo.com/](https://example-app.odoo.com/)
2. The Database Name is the specific name of the database you wish to connect. It can be found in the URL subdomain, such as `example-app` in the URL [https://example-app.odoo.com/](https://example-app.odoo.com/).
3. User Name is the username of the service account.
4. Password is the password for the service account user.
## Connecting Odoo to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Odoo" as your data source.
4. Enter a name for your data source and input credentials you obtained in the previous step.
5. Click "Next" to establish the data source.
![]()
Congratulations! You've successfully connected Odoo to Upsolve. You can now access and manage your Odoo data seamlessly within the Upsolve platform.
# OpenAI
Source: https://docs.upsolve.ai/integrations/openai
Discover a seamless integration process! Learn how to connect OpenAI to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting OpenAI to Upsolve, allowing you to seamlessly access your OpenAI data within the Upsolve platform.
## Prerequisites
Before you start, you will need an API Key from OpenAI.
Follow the steps below to obtain it:
1. Go to this [link](https://platform.openai.com/account/api-keys) to create or obtain your OpenAI API Key.
2. Click the "Create new secret key" button to generate your API Key.
![]()
## Connecting OpenAI to Upsolve
Now that you have your requirements, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "OpenAI" as your data source.
4. Enter your data source name and API Key.
![]()
5. Click "Authorize" to create your OpenAI data source.
Congratulations! You've successfully created a new data source between OpenAI and Upsolve. You can now access and manage your OpenAI data seamlessly within the Upsolve platform.
# Oracle
Source: https://docs.upsolve.ai/integrations/oracle
Discover a seamless integration process! Learn how to connect Oracle to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Oracle to Upsolve, allowing you to seamlessly access your Oracle data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Oracle.
* Host (Endpoint without the port or database name)
* Port (The port used to connect to your database)
* User (The username of the service account)
* Password (The password for the service account user)
* Database Name (The database name of your service)
* SSL, if active (Toggle "Connect Through TLS" switch)
## Connecting Oracle to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Oracle" as your data source.
4. Fill out the data source details form as shown in the image below, including:
* Host (Endpoint without the port or database name)
* Port (The port used to connect to your database)
* User (The username of the service account)
* Password (The password for the service account user)
* Database Name (The database name of your service)
* Connect Through TLS (Toggle switch if SSL is active)
5. Click "Next" to establish the data source.
![]()
Congratulations! You've successfully created a data source between Oracle and Upsolve. You can now access and manage your Oracle data seamlessly within the Upsolve platform.
# Pinecone
Source: https://docs.upsolve.ai/integrations/pinecone
Discover a seamless integration process! Learn how to connect Pinecone to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Pinecone to Upsolve, allowing you to seamlessly access your Pinecone data within the Upsolve platform.
## Prerequisites
Pinecone offers two distinct connection methods:
* Pinecone Connect
* API Key
Before you start, you will need the following information from Pinecone.
* API Key (if required for the connection method)
Follow the steps below to obtain it:
1. To obtain API Key, check this [article](https://docs.pinecone.io/guides/get-started/authentication).
## Connecting Pinecone to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Pinecone to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "ActiveCampaign" as your data source.
4. Choose the authentication method that best suits your needs.
5. If you're using the API Key connection method, simply enter your API Key and click the Authorize button.
6. If you're using the Pinecone Connect method, click the Connect to Pinecone button and allow Upsolve to access your Pinecone data.
![]()
Congratulations! You've successfully connected Pinecone to Upsolve. You can now access and manage your Pinecone data seamlessly within the Upsolve platform.
# Pipedrive
Source: https://docs.upsolve.ai/integrations/pipedrive
Discover a seamless integration process! Learn how to connect Pipedrive to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Pipedrive to Upsolve, allowing you to seamlessly access your Pipedrive data within the Upsolve platform.
## Prerequisites
Before you start, you will need an API Key from Pipedrive.
Follow the steps below to obtain it:
1. Visit the [Pipedrive website](https://app.pipedrive.com/).
2. It can be found under **Settings > Personal preferences > API**.
![]()
If you cannot find your API token here, it may be that the permission set you
are in does not have that permission enabled.
## Connecting Pipedrive to Upsolve
Now that you have your Pipedrive API Key, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Pipedrive" as your data source.
4. Enter a name for your data source name and paste the Pipedrive API Key into the "API Key" input field.
![]()
5. Click "Authorize" to create your Pipedrive data source.
Congratulations! You've successfully connected Pipedrive to Upsolve. You can now access and manage your Pipedrive data seamlessly within the Upsolve platform.
# PostgreSQL
Source: https://docs.upsolve.ai/integrations/postgresql
Discover a seamless integration process! Learn how to connect PostgreSQL to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting PostgreSQL to Upsolve, allowing you to seamlessly access your PostgreSQL data within the Upsolve platform.
Note: Users can establish a data source to their PostgreSQL database on AWS by leveraging IAM credentials. For more details, refer to the [article on Upsolve's documentation](https://www.upsolve.com/docs/how-to-guides/how-to-connect-databases-in-aws-using-iam-credentials/).
## Prerequisites
Before you start, you will need the following information from PostgreSQL.
* Host (Endpoint without the port or database name)
* Port (The port used to connect to your database)
* User (The username of the service account)
* Password (The password for the service account user)
* Database Name (The database name of your service)
* Schema (The name of your schema)
* SSL, if active (Toggle "Connect Through TLS" switch)
## Connecting PostgreSQL to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "PostgreSQL" as your data source.
4. Fill out the data source details form as shown in the image below, including:
* Host (Endpoint without the port or database name)
* Port (The port used to connect to your database)
* User (The username of the service account)
* Password (The password for the service account user)
* Database Name (The database name of your service)
* Schema (The name of your schema)
* Connect Through TLS (Toggle switch if SSL is active)
5. Click "Next" to establish the data source.
![]()
Congratulations! You've successfully created a new data source between PostgreSQL and Upsolve. You can now access and manage your PostgreSQL data seamlessly within the Upsolve platform.
# ProfitWell
Source: https://docs.upsolve.ai/integrations/profitwell
Discover a seamless integration process! Learn how to connect ProfitWell to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting ProfitWell to Upsolve, allowing you to seamlessly access your ProfitWell data within the Upsolve platform.
## Prerequisites
Before you start, you will need an API Key from ProfitWell.
Follow the steps below to obtain it:
1. Go to this [link](https://www2.profitwell.com/app/account/integrations) to create or obtain your ProfitWell API Key or private token.
2. Copy your private token for use in the integration.
## Connecting ProfitWell to Upsolve
Now that you have your requirements, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "ProfitWell" as your data source.
4. Enter your data source name and API Key.
![]()
5. Click "Authorize" to create your ProfitWell data source.
Congratulations! You've successfully created a new data source between ProfitWell and Upsolve. You can now access and manage your ProfitWell data seamlessly within the Upsolve platform.
# Provet Cloud
Source: https://docs.upsolve.ai/integrations/provet-cloud
Discover a seamless integration process! Learn how to connect Provet Cloud to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Provet Cloud to Upsolve, allowing you to seamlessly access your Provet Cloud data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Provet Cloud.
* Client ID
* Client Secret
* Provet ID
* Environment
Follow the steps below to obtain it:
1. To obtain Client ID and Client Secret, check this [article](https://developers.provetcloud.com/restapi/authentication_oauth2.html#client-credentials-example-get-access-token).
2. To obtain Provet ID, check this [article](https://support.provet.cloud/hc/en-gb/articles/360009783098-Logging-In-to-Provet-Cloud).
3. To obtain Environment, check this [article](https://support.provet.cloud/hc/en-gb/articles/360010249038-Integrating-Third-party-Applications-with-Provet-Cloud).
## Connecting Provet Cloud to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Provet Cloud to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Provet Cloud" as your data source.
4. Enter a name for your data source, input the credentials you obtained in the previous steps.
![]()
5. Click "Authorize" to create your Provet Cloud data source.
Congratulations! You've successfully connected Provet Cloud to Upsolve. You can now access and manage your Provet Cloud data seamlessly within the Upsolve platform.
# Qdrant
Source: https://docs.upsolve.ai/integrations/qdrant
Discover a seamless integration process! Learn how to connect Qdrant to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Qdrant to Upsolve, allowing you to seamlessly access your Qdrant data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Qdrant.
* URL
* API Key
Follow the steps below to obtain it:
1. To obtain URL and API Key, check this [article](https://qdrant.tech/documentation/cloud/authentication/#).
## Connecting Qdrant to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Qdrant to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Qdrant" as your data source.
4. Enter a name for your data source, input the credentials you obtained in the previous steps.
![]()
5. Click "Authorize" to create your Qdrant data source.
Congratulations! You've successfully connected Qdrant to Upsolve. You can now access and manage your Qdrant data seamlessly within the Upsolve platform.
# Quickbooks Online
Source: https://docs.upsolve.ai/integrations/quickbooks-online
Discover a seamless integration process! Learn how to connect Quickbooks Online to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Quickbooks Online to Upsolve, allowing you to seamlessly access your Quickbooks Online data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Quickbooks Online.
* Client ID (required for advanced authentication)
* Client Secret (required for advanced authentication)
Follow the steps below to obtain it:
1. To obtain the Client ID and Client Secret, go to the [article](https://quickbooks.intuit.com/learn-support/en-us/account-management/how-to-get-client-id-and-client-secret-from-my-qb-online-account/00/928730).
## Connecting Quickbooks Online to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Quickbooks Online" as your data source.
4. Enter a name for your data source and input credentials you obtained in the previous step.
5. Click "Sign in" to create your Quickbooks Online data source and allow Upsolve access to your Quickbooks Online account.
![]()
Congratulations! You've successfully connected Quickbooks Online to Upsolve. You can now access and manage your Quickbooks Online data seamlessly within the Upsolve platform.
# Salesforce
Source: https://docs.upsolve.ai/integrations/salesforce
Discover a seamless integration process! Learn how to connect Salesforce to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Salesforce to Upsolve, allowing you to seamlessly access your Salesforce data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Salesforce.
* User Name
* Password
* Security Token
Follow the steps below to obtain it:
1. To obtain the User Name and Password, go to the [article](https://support.omaticsoftware.com/s/article/Locating-your-Salesforce-Username-and-Password).
2. To obtain the Security Token, log in to your Salesforce account.
3. Click the profile avatar and choose Settings.
![]()
4. Select My Personal Information → Reset My Security Token.
![]()
5. Check your email for the security token.
![]()
## Connecting Salesforce to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Salesforce" as your data source.
4. Enter a name for your data source and input credentials you obtained in the previous step.
5. Click "Next" to create your Salesforce data source.
![]()
Congratulations! You've successfully connected Salesforce to Upsolve. You can now access and manage your Salesforce data seamlessly within the Upsolve platform.
# SAP HANA
Source: https://docs.upsolve.ai/integrations/sap-hana
Discover a seamless integration process! Learn how to connect SAP HANA to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting SAP HANA to Upsolve, allowing you to seamlessly access your SAP HANA data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from SAP HANA.
* Host (endpoint without the port or database name)
* Port (the port used to connect to your database)
* User (the username of the service account)
* Password (the password for the service account user)
* Database Name (the database name of your service)
* SSL, if active (toggle "Connect through TLS" switch)
* Show System Schemas (enable the switch to display style schemas)
## Connecting SAP HANA to Upsolve
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "SAP Hana" as your data source.
4. Fill out the data source details form as shown in the image below, including:
* Host (endpoint without the port or database name)
* Port (the port used to connect to your database)
* User (the username of the service account)
* Password (the password for the service account user)
* Database Name (the database name of your service)
* Connect Through TLS (toggle switch if SSL is active)
* Show System Schemas (enable its switch to display style schemas)
5. Click "Next" to establish the data source.
![]()
Congratulations! You've successfully created a new data source between SAP HANA and Upsolve. You can now access and manage your SAP HANA data seamlessly within the Upsolve platform.
# Segment
Source: https://docs.upsolve.ai/integrations/segment
Discover a seamless integration process! Learn how to connect Segment to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Segment to Upsolve, allowing you to seamlessly access your Segment data within the Upsolve platform.
## Prerequisites
Before you start, you will need an API Key from Segment.
Follow the steps below to obtain it:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Segment" as your data source.
4. Input the necessary information, including the Segment Catalog Name, and proceed by clicking the "Next" button.
![]()
5. Upon clicking the "Next" button, your unique API Key will be automatically generated. Please note that the API Key will only be displayed once.
![]()
6. Copy the generated API Key from Upsolve and paste it into the designated field within your Segment account settings.
## Adding Upsolve Destination in Segment
To integrate Upsolve with Segment, follow these steps to paste your API Key:
1. Navigate to the Upsolve Destination page on Segment by clicking on the following link:
[https://app.segment.com/\{your-workspace-name}/destinations/catalog/upsolve](https://app.segment.com/\{your-workspace-name}/destinations/catalog/upsolve)
2. On the Upsolve Destination page, locate and click on the "Add Destination" button.
![]()
3. In the dialogue box that appears, provide a name for your destination and proceed by clicking on the "Create Destination" button.
![]()
4. Once the destination is created, locate the designated field for API Key entry. Paste the API Key obtained from Upsolve into this field.
![]()
5. After pasting the API Key, make sure to save your changes to finalize the integration.
# SendGrid
Source: https://docs.upsolve.ai/integrations/sendgrid
Discover a seamless integration process! Learn how to connect SendGrid to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting SendGrid to Upsolve, allowing you to seamlessly access your SendGrid data within the Upsolve platform.
## Prerequisites
Before you start, you will need an API Key from Open AI.
Follow the steps below to obtain it:
1. Go to this [link](https://www2.Sendgrid.com/app/account/integrations) to create or obtain your SendGrid API Key.
2. Click the "Create API Key" button to generate your API Key.
![]()
## Connecting SendGrid to Upsolve
Now that you have your requirements, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Sendgrid" as your data source.
4. Enter your data source name and API Key.
![]()
5. Click "Authorize" to create your SendGrid data source
Congratulations! You've successfully created a data source between SendGrid and Upsolve. You can now access and manage your SendGrid data seamlessly within the Upsolve platform.
# ServiceTitan
Source: https://docs.upsolve.ai/integrations/service-titan
Discover a seamless integration process! Learn how to connect ServiceTitan to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting ServiceTitan to Upsolve, allowing you to seamlessly access your ServiceTitan data within the Upsolve platform.
## Prerequisites
Before you begin, ensure you have the following credentials from ServiceTitan:
* **Client ID**
* **Client Secret**
* **Tenant**
* **Application Key**
Follow these steps to obtain them:
1. To get the Client ID and Client Secret, refer to [this article](https://developer.servicetitan.io/docs/get-going-manage-client-id-and-secret/).
2. To get the Tenant, follow these steps:
* Click the gear icon ⚙️ in the top-right corner.
* Navigate to **Integrations > API Application Access** in the left-hand menu. Your Tenant ID will be displayed on the right. Copy it.
3. To get the Application Key, follow these steps:
1. Log in to your [ServiceTitan Developer account](https://developer.servicetitan.io/signin/). Select either the integration or production environment, depending on which environment your app will be used in.
2. Click the + New App button.
3. Enter all **App details** for your custom app, including the **Tenant(s)** that will use the custom app and the necessary **API Scopes**. For additional details, refer to the [ServiceTitan Create an App API documentation](https://developer.servicetitan.io/docs/get-going-create-an-app/).
4. Click the Create App button. You will return to the **My Apps** page.
5. Next to your new app, click **Edit**.
6. Copy the Application Key.
## Connecting ServiceTitan to Upsolve
Now that you have your ServiceTitan Client ID, Clint Secret, Tenant and Application Key, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "ServiceTitan" as your data source.
4. Enter a name for your data source name and paste values that you before.
![]()
5. Click "Authorize" to create your ServiceTitan data source.
Congratulations! You've successfully connected ServiceTitan to Upsolve. You can now access and manage your ServiceTitan data seamlessly within the Upsolve platform.
# Shopify
Source: https://docs.upsolve.ai/integrations/shopify
Discover a seamless integration process! Learn how to connect Shopify to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Shopify to Upsolve, allowing you to seamlessly access your Shopify data within the Upsolve platform.
## Prerequisites
Before you start, you will need Admin API Access Token and Subdomain from Shopify.
Follow the steps below to obtain it:
1. Go to this [link](https://www.shopify.com/) and log in to your Shopify account.
2. Click the settings icon located in the lower-left corner of the website.
3. If you already have a Shopify app, select it. If not, create a new app.
1. Click the "Develop apps" button, as shown in the image below.
1. Configure your Admin API scopes and Storefront API scopes as needed for your integration.
1. After configuring your API scopes, click "Install app" button to generate Admin API Access Token.
1. Copy the Access Token generated for your app.
1. Go to the "Domains" section on the side panel of the website to obtain or connect your Shopify subdomain.
## Connecting Shopify to Upsolve
Now that you have your requirements, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Shopify" as your data source
4. Enter your data source name, Admin API Access Token and Subdomain.\\
![Screenshot 2025-04-29 at 16.12.07.png]()
5. Click "Authorize" to create your Shopify data source.
Congratulations! You've successfully created a data source between Shopify and Upsolve. You can now access and manage your Shopify data seamlessly within the Upsolve platform.
# SingleStore
Source: https://docs.upsolve.ai/integrations/single-store
Discover a seamless integration process! Learn how to connect SingleStore to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting SingleStore to Upsolve, allowing you to seamlessly access your SingleStore data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from SingleStore.
* User
* Password
* Host
* Port
* Database Name
Follow the steps below to obtain it:
1. Go to the SingleStore website and log in to your account if you're not already logged in.
2. Create a new workspace on SingleStore. Please note that this step may take some time.
3. Click the "Connect" button and select "Connect Directly." This will take you to the SingleStore dashboard.
![]()
4. Click on the "SQL IDE" tab. Here, you will find your user name, port, and host.
![]()
5. Select the "Access" tab. Here, you will find your password. You can reset your password if needed.
![]()
6. Select the "Database" tab and copy your database name.
## Connecting SingleStore to Upsolve
Now that you have your requirements, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "SingleStore" as your data source
4. Enter your data source name, user, password, host, port, and database name. Optionally, enable the "Connect through TLS" switch if SSL is active.
![]()
5. Click "Authorize" to create your SingleStore data source.
![]()
Congratulations! You've successfully created a data source between SingleStore and Upsolve. You can now access and manage your SingleStore data seamlessly within the Upsolve platform.
# Slack
Source: https://docs.upsolve.ai/integrations/slack
Discover a seamless integration process! Learn how to connect Slack to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Slack to Upsolve, allowing you to seamlessly access your Slack data within the Upsolve platform.
Follow these detailed steps to establish a connection between Slack and Upsolve:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Slack" as your data source.
4. Enter a name for your data source.
![]()
5. Click the "Signing in to Slack" button. This will open a modal to allow Upsolve access to your Slack account.
6. It will open a modal to grant permission to Upsolve.
![]()
Congratulations! You've successfully created a new data source between Slack and Upsolve. You can now access and manage your Slack data seamlessly within the Upsolve platform.
# SnowFlake
Source: https://docs.upsolve.ai/integrations/snowflake
Discover a seamless integration process! Learn how to connect SnowFlake to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting SnowFlake to Upsolve, allowing you to seamlessly access your SnowFlake data within the Upsolve platform.
## Prerequisites
Before you begin, ensure you have the following credentials from ServiceTitan:
* **Account**
* **User**
* **Password**
* **Database Name**
* **Role**
* **Warehouse**
* **Http Proxy**
Follow the steps below to obtain it:
1. To get the Account information, follow these steps:
* Log in to your Snowflake account. If you are logging in for the first time, use the email you have received from Snowflake.
* At the bottom left, click on your generated account name next to the Snowflake icon.
* Copy the **Account** name shown and store them in a safe place.
2. To get the User and Password information, follow these steps:
* User and Password fields are your Snowflake account's login credentials.
3. To get the Database Name information, follow these steps:
* This is the name of your database before creation. If you haven't created a database yet, you will need to do so.
4. To get the Role information, follow these steps:
* Navigate to **Admin > Users & Roles** in the left-hand menu.
* Select the **Roles** tab, and your roles will be displayed. Copy the necessary role.
5. To get the Warehouse information, follow these steps:
* Navigate to **Admin > Warehouses** in the left-hand menu.
* Your warehouses will be shown. Copy the required warehouse.
6. To get the Http Proxy information, follow these steps:
* If you need to set up an HTTP Proxy, refer to the following articles:
* [AWS](https://docs.snowflake.com/en/sql-reference/external-functions-creating-aws-ui-proxy-service)
* [Google Cloud](https://docs.snowflake.com/en/sql-reference/external-functions-creating-gcp-ui-proxy-service)
## Connecting SnowFlake to Upsolve
Now that you have your SnowFlake Account, User,Password, Database Name, Role, Warehouse and Http Proxy, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "SnowFlake" as your data source.
4. Enter a name for your data source name and paste values that you before.
![]()
5. Click "Next" to create your SnowFlake data source.
Congratulations! You've successfully connected SnowFlake to Upsolve. You can now access and manage your SnowFlake data seamlessly within the Upsolve platform.
# Stripe
Source: https://docs.upsolve.ai/integrations/stripe
Discover a seamless integration process! Learn how to connect Stripe to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Stripe to Upsolve, allowing you to seamlessly access your Stripe data within the Upsolve platform.
## Prerequisites
Before you start, you will need an API Key from Stripe.
Follow the steps below to obtain it:
1. Go to this [link](https://stripe.com/) and log in to your Stripe account. This will take you to your Stripe dashboard.
2. Click the "Developers" button, as shown in the image below.
![]()
3. In the Developers section, click the "API Keys" button.Then click the "Reveal test key" button to display your API Key.
![]()
## Connecting Stripe to Upsolve
Now that you have your requirements, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Stripe" as your data source.
4. Enter your data source name and API Key.
![]()
5. Click "Authorize" to create your Stripe data source.
Congratulations! You've successfully created a data source between Stripe and Upsolve. You can now access and manage your Stripe data seamlessly within the Upsolve platform.
# Twilio
Source: https://docs.upsolve.ai/integrations/twilio
Discover a seamless integration process! Learn how to connect Twilio to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Twilio to Upsolve, allowing you to seamlessly access your Twilio data within the Upsolve platform.
## Prerequisites
Before you start, you will need SID and Auth Token from Twilio.
Follow the steps below to obtain it:
1. Go to this [link](https://console.twilio.com/).
2. Scroll down on the page, and you will find your Twilio SID and Auth Token, as shown in the image below.
![]()
## Connecting Twilio to Upsolve
Now that you have your requirements, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Twilio" as your connection.
4. Enter a name for your data source name and paste values that you got before.
![]()
5. Click "Authorize" to create your Twilio data source.
Congratulations! You've successfully created a data source between Twilio and Upsolve. You can now access and manage your Twilio data seamlessly within the Upsolve platform.
# Upstash Vector
Source: https://docs.upsolve.ai/integrations/upstash-vector
Discover a seamless integration process! Learn how to connect Upstash Vector to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Upstash Vector to Upsolve, allowing you to seamlessly access your Upstash Vector data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Upstash Vector.
* Index Endpoint
* Token
Follow the steps below to obtain it:
1. To obtain your Index Endpoint and Token, navigate to the project dashboard. If you don't have an existing Upstash Vector project, create a new one to access these credentials.
![]()
## Connecting Upstash Vector to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Upstash Vector to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Upstash Vector" as your data source.
4. Enter a name for your data source and paste values that you got before.
![]()
5. Click "Authorize" to create your Upstash Vector data source.
Congratulations! You've successfully connected Upstash Vector to Upsolve. You can now access and manage your Upstash Vector data seamlessly within the Upsolve platform.
# Weaviate
Source: https://docs.upsolve.ai/integrations/weaviate
Discover a seamless integration process! Learn how to connect Weaviate to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Weaviate to Upsolve, allowing you to seamlessly access your Weaviate data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Weaviate.
* Weaviate Server URL
* API Key
Follow the steps below to obtain it:
1. To obtain Weaviate Server URL and API Key, check this [article](https://weaviate.io/developers/weaviate/quickstart#:~:text=You%20need%20a%20Weaviate%20instance,the%20Details%20tab%20in%20WCD.).
## Connecting Weaviate to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Weaviate to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Weaviate" as your data source.
4. Enter a name for your data source and paste values that you got before.
![]()
5. Click "Authorize" to create your Weaviate data source.
Congratulations! You've successfully connected Weaviate to Upsolve. You can now access and manage your Weaviate data seamlessly within the Upsolve platform.
# Youtube Analytics
Source: https://docs.upsolve.ai/integrations/youtube-analytics
Discover a seamless integration process! Learn how to connect Youtube Analytics to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Youtube Analytics to Upsolve, allowing you to seamlessly access your Youtube Analytics data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Youtube Analytics.
* Client ID (required for advanced authentication)
* Client Secret (required for advanced authentication)
Follow the steps below to obtain it:
1. To obtain the Client ID and Client Secret, refer to this [article](https://developers.google.com/identity/protocols/oauth2#1.-obtain-oauth-2.0-credentials-from-the-dynamic_data.setvar.console_name-.).
## Connecting Youtube Analytics to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Youtube Analytics to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Youtube Analytics" as your data source.
4. Enter a name for your data source and paste values that you got before.
![]()
5. Click "Sign in" to create your Youtube Analytics data source and allow the connection to access your Youtube Analytics data.
Congratulations! You've successfully connected Youtube Analytics to Upsolve. You can now access and manage your Youtube Analytics data seamlessly within the Upsolve platform.
# Zendesk
Source: https://docs.upsolve.ai/integrations/zendesk
Discover a seamless integration process! Learn how to connect Zendesk to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Zendesk to Upsolve, allowing you to seamlessly access your Zendesk data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Zendesk.
* Domain
* Unique Identifier
* Secret
Follow the steps below to obtain it:
1. Go to [Zendesk website](https://www.zendesk.com) and sign in to your Zendesk account.
2. Click on the settings icon, as shown in the image below, to access your account settings.
![]()
3. On the Settings page, you will find your Zendesk domain. Make a note of it.
![]()
4. Navigate to the Admin Center by clicking on it, as shown in the image below.
![]()
5. Inside the Admin Center, go to the "Zendesk API" tab from the left menu and click the "Add OAuth Client" button.
![]()
6. Begin by entering a distinct name and identifier for your integration. Following that, choose "Confidential" as the Client type. Proceed by inserting the following URL for the Redirect URLs parameter:
[https://upsolve.studio/oauth2/callback](https://upsolve.studio/oauth2/callback)
![]()
7. Next, click the save button to establish your OAuth client. Upon creation, Zendesk will produce a Secret token for you. It's crucial to record this for future reference.
![]()
## Connecting Zendesk to Upsolve
Now that you have your requirements, you can connect it to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Zendesk" as your data source.
4. Enter a name for your data source and paste values that you got before.
![]()
5. Click "Authorize" to create your Zendesk data source.
Congratulations! You've successfully created a new data source between Zendesk and Upsolve. You can now access and manage your Zendesk data seamlessly within the Upsolve platform.
# Zoho CRM
Source: https://docs.upsolve.ai/integrations/zoho-crm
Discover a seamless integration process! Learn how to connect Zoho CRM to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Zoho CRM to Upsolve, allowing you to seamlessly access your Zoho CRM data within the Upsolve platform.
Follow these steps to connect Zoho CRM to Upsolve:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Zoho CRM" as your data source.
4. Enter a name for your data source and select a domain.
5. Click the "Signing in to Zoho CRM" button. This will open a modal to allow Upsolve access to your Zoho CRM account.
![]()
6. In the modal, click the "Allow" button to grant Upsolve access to your Zoho CRM data.
![]()
Congratulations! You've successfully connected Zoho CRM to Upsolve. You can now access and manage your Zoho CRM data seamlessly within the Upsolve platform.
# Zoom
Source: https://docs.upsolve.ai/integrations/zoom
Discover a seamless integration process! Learn how to connect Zoom to Upsolve effortlessly for streamlined transactions.
This article guides you through the process of connecting Zoom to Upsolve, allowing you to seamlessly access your Zoom data within the Upsolve platform.
## Prerequisites
Before you start, you will need the following information from Zoom.
* Client ID (required for advanced authentication)
* Client Secret (required for advanced authentication)
Follow the steps below to obtain it:
1. Navigate to the [Zoom Marketplace](https://marketplace.zoom.us/) and select 'Build App' from the dropdown menu to begin the app creation process.
![]()
2. Choose the type of app you want to create from the available options.
![]()
3. In the left-side menu, locate the section highlighted in a red rectangle and copy your Client ID and Client Secret from there.
![]()
4. Scroll down to the "Select how the app is managed" section and select "Admin-managed" to specify the management type for your app.
![]()
5. Scroll down to the "OAuth Information" section, where you will find the field for the OAuth Redirect URL. Enter `https://upsolve.studio/oauth2/callback` in this field, as it is crucial for redirecting users after they complete the authentication process.
![]()
6. In the left-side menu, click on the "Scopes" option, then click the "Add Scopes" button to proceed with defining the permissions your app will require.
7. In the modal that appears, choose the following mandatory scopes to ensure your app has the necessary permissions for optimal functionality:
* View users
* View a user's meetings
* View a past meeting's instances
* View a past meeting's participants
* View a meeting's registrants
* View a user's upcoming meetings
* View a meeting
* View a meeting's participant
* View a past meeting
* View a meeting's registrant
![]()
8. After reviewing your selected scopes, click the "Continue" button.
## Connecting Zoom to Upsolve
Now that you have obtained the necessary integration data and credentials, you can connect Zoom to Upsolve using the following steps:
1. Navigate to the data model page in Upsolve.
2. In the side menu bar, click the "New Data Source" button to initiate the connection process.
3. In the modal that appears, select "Zoom" as your data source.
4. Enter a name for your data source and paste values that you got before.
![]()
5. Click "Sign In" to connect Zoom and allow access to your data.
Congratulations! You've successfully connected Zoom to Upsolve. You can now access and manage your Zoom data seamlessly within the Upsolve platform.
# Agent Studio for Data Teams
Source: https://docs.upsolve.ai/introduction
Build analytics agents that know your business — not just your schema.
![Hero Light]()
![Hero Dark]()
## The problem every data team hits
Your KPIs are documented in Notion. Your business logic lives in a SQL file from 2021. Your team has spent years building institutional knowledge about what the numbers actually mean — what counts as "revenue," which user segments to exclude, when to apply which filter.
Generic AI tools know none of it. Connect them to your database and they'll generate SQL — but not the *right* SQL for your specific data model, with your specific business rules applied correctly. The answers look right, but aren't. And that destroys trust faster than having no AI at all.
## What Upsolve is
Upsolve is where data teams encode that institutional knowledge into a reliable analytics agent, then expose it to the whole business.
It's a two-sided platform. On one side, data experts use **Agent Studio** to structure everything the agent needs to know: which tables matter, what your KPIs actually mean, examples of correct answers to common questions, and rules for who sees what data. On the other side, business users get an analytics interface that answers their questions using that full context — accurately, and without involving the data team every time.
## What Agent Studio is
Agent Studio is the workspace where you build and maintain that context. It's organized around the idea that a reliable agent isn't just a model connected to a database — it's a model that has been deliberately taught your business:
**Encode your context** — Define which tables and columns the agent can see, annotate them with business meaning, write the rules it should always follow, and give it examples of how your team actually answers common questions. This is the core work, and it's what makes the agent trustworthy rather than just capable.
**Deploy and expose** — Once the agent is configured and tested, deploy it to your product. Agent Studio handles multi-tenancy, row-level security, and the embedding layer so every user sees only their own data through a clean analytics interface.
**Improve over time** — Every question the agent handles shows you where its context is complete and where it's missing. The feedback loop is built in: gaps surface as eval failures, and fixing them means adding more context, not retraining a model.
## Get started
Connect your data and get your first agent running.
End-to-end walkthrough from first connection to production deployment.
## Encoding your context
The three pages below are the core of Agent Studio. Work through them in order when building a new agent.
Choose which tables and columns the agent can see, and annotate them with business meaning.
Encode business rules, KPI definitions, and behavioral guardrails.
Add example queries and charts the agent references for common question patterns.
## Deploying to your users
The embeddable dashboard and chart interface that delivers your agent's output to end users.
Embed your agent natively in your product with backend auth and frontend setup guides.
# Aggregate functions
Source: https://docs.upsolve.ai/sql/functions/aggregate
Aggregate functions operate on a set of values to compute a single result.
Except for `count`, `count_if`, `max_by`, `min_by` and
`approx_distinct`, all of these aggregate functions ignore null values
and return null for no input rows or when all values are null. For example,
`sum` returns null rather than zero and `avg` does not include null
values in the count. The `coalesce` function can be used to convert null into
zero.
## Ordering during aggregation
Some aggregate functions such as `array_agg` produce different results
depending on the order of input values. This ordering can be specified by writing
an order by clause within the aggregate function:
```sql theme={null}
array_agg(x ORDER BY y DESC)
array_agg(x ORDER BY x, y, z)
```
## Filtering during aggregation
The `FILTER` keyword can be used to remove rows from aggregation processing
with a condition expressed using a `WHERE` clause. This is evaluated for each
row before it is used in the aggregation and is supported for all aggregate
functions.
```sql theme={null}
aggregate_function(...) FILTER (WHERE )
```
A common and very useful example is to use `FILTER` to remove nulls from
consideration when using `array_agg`
```sql theme={null}
SELECT array_agg(name) FILTER (WHERE name IS NOT NULL)
FROM region;
```
As another example, imagine you want to add a condition on the count for Iris
flowers, modifying the following query:
```sql theme={null}
SELECT species,
count(*) AS count
FROM iris
GROUP BY species;
```
```
species | count
-----------+-------
setosa | 50
virginica | 50
versicolor | 50
```
If you just use a normal `WHERE` statement you lose information:
```sql theme={null}
SELECT species,
count(*) AS count
FROM iris
WHERE petal_length_cm > 4
GROUP BY species;
```
```
species | count
-----------+-------
virginica | 50
versicolor | 34
```
Using a filter you retain all information:
```sql theme={null}
SELECT species,
count(*) FILTER (where petal_length_cm > 4) AS count
FROM iris
GROUP BY species;
```
```
species | count
-----------+-------
virginica | 50
setosa | 0
versicolor | 34
```
## General aggregate functions
### any\_value
```
any_value(x) -> [same as input]
```
Returns an arbitrary non-null value `x`, if one exists. `x` can be any
valid expression. This allows you to return values from columns that are not
directly part of the aggregation, inluding expressions using these columns,
in a query.
For example, the following query returns the customer name from the `name`
column, and returns the sum of all total prices as customer spend. The
aggregation however uses the rows grouped by the customer identifier
`custkey` a required, since only that column is guaranteed to be unique:
```sql theme={null}
SELECT sum(o.totalprice) as spend,
any_value(c.name)
FROM tpch.tiny.orders o
JOIN tpch.tiny.customer c
ON o.custkey = c.custkey
GROUP BY c.custkey;
ORDER BY spend;
```
### arbitrary
```
arbitrary(x) -> [same as input]
```
Returns an arbitrary non-null value of `x`, if one exists. Identical to [any\_value](#any-value).
### array\_agg
```
array_agg(x) -> array<[same as input]>
```
Returns an array created from the input `x` elements.
### avg
```
avg(x) -> double
```
Returns the average (arithmetic mean) of all input values.
```
avg(time interval type) -> time interval type
```
Returns the average interval length of all input values.
### bool\_and
```
bool_and(boolean) -> boolean
```
Returns `TRUE` if every input value is `TRUE`, otherwise `FALSE`.
### bool\_or
```
bool_or(boolean) -> boolean
```
Returns `TRUE` if any input value is `TRUE`, otherwise `FALSE`.
### checksum
```
checksum(x) -> varbinary
```
Returns an order-insensitive checksum of the given values.
### count
```
count(*) -> bigint
```
Returns the number of input rows.
```
count(x) -> bigint
```
Returns the number of non-null input values.
### count\_if
```
count_if(x) -> bigint
```
Returns the number of `TRUE` input values.
This function is equivalent to `count(CASE WHEN x THEN 1 END)`.
### every
```
every(boolean) -> boolean
```
This is an alias for `bool_and`.
### geometric\_mean
```
geometric_mean(x) -> double
```
Returns the geometric mean of all input values.
### listagg
```
listagg(x, separator) -> varchar
```
Returns the concatenated input values, separated by the `separator` string.
Synopsis:
```sql theme={null}
LISTAGG( expression [, separator] [ON OVERFLOW overflow_behaviour])
WITHIN GROUP (ORDER BY sort_item, [...]) [FILTER (WHERE condition)]
```
If `separator` is not specified, the empty string will be used as `separator`.
In its simplest form the function looks like:
```sql theme={null}
SELECT listagg(value, ',') WITHIN GROUP (ORDER BY value) csv_value
FROM (VALUES 'a', 'c', 'b') t(value);
```
and results in:
```
csv_value
-----------
'a,b,c'
```
The overflow behaviour is by default to throw an error in case that the length of the output
of the function exceeds `1048576` bytes:
```sql theme={null}
SELECT listagg(value, ',' ON OVERFLOW ERROR) WITHIN GROUP (ORDER BY value) csv_value
FROM (VALUES 'a', 'b', 'c') t(value);
```
There exists also the possibility to truncate the output `WITH COUNT` or `WITHOUT COUNT`
of omitted non-null values in case that the length of the output of the
function exceeds `1048576` bytes:
```sql theme={null}
SELECT listagg(value, ',' ON OVERFLOW TRUNCATE '.....' WITH COUNT) WITHIN GROUP (ORDER BY value)
FROM (VALUES 'a', 'b', 'c') t(value);
```
If not specified, the truncation filler string is by default `'...'`.
This aggregation function can be also used in a scenario involving grouping:
```sql theme={null}
SELECT id, listagg(value, ',') WITHIN GROUP (ORDER BY o) csv_value
FROM (VALUES
(100, 1, 'a'),
(200, 3, 'c'),
(200, 2, 'b')
) t(id, o, value)
GROUP BY id
ORDER BY id;
```
results in:
```
id | csv_value
-----+-----------
100 | a
200 | b,c
```
This aggregation function supports
[filtering during aggregation](#aggregate-function-filtering-during-aggregation)
for scenarios where the aggregation for the data not matching the filter
condition still needs to show up in the output:
```sql theme={null}
SELECT
country,
listagg(city, ',')
WITHIN GROUP (ORDER BY population DESC)
FILTER (WHERE population >= 10_000_000) megacities
FROM (VALUES
('India', 'Bangalore', 13_700_000),
('India', 'Chennai', 12_200_000),
('India', 'Ranchi', 1_547_000),
('Austria', 'Vienna', 1_897_000),
('Poland', 'Warsaw', 1_765_000)
) t(country, city, population)
GROUP BY country
ORDER BY country;
```
results in:
```
country | megacities
---------+-------------------
Austria | NULL
India | Bangalore,Chennai
Poland | NULL
```
The current implementation of `listagg` function does not support window frames.
### max
```
max(x) -> [same as input]
```
Returns the maximum value of all input values.
```
max(x, n) -> array<[same as x]>
```
Returns `n` largest values of all input values of `x`.
### max\_by
```
max_by(x, y) -> [same as x]
```
Returns the value of `x` associated with the maximum value of `y` over all input values.
```
max_by(x, y, n) -> array<[same as x]>
```
Returns `n` values of `x` associated with the `n` largest of all input values of `y`
in descending order of `y`.
### min
```
min(x) -> [same as input]
```
Returns the minimum value of all input values.
```
min(x, n) -> array<[same as x]>
```
Returns `n` smallest values of all input values of `x`.
### min\_by
```
min_by(x, y) -> [same as x]
```
Returns the value of `x` associated with the minimum value of `y` over all input values.
```
min_by(x, y, n) -> array<[same as x]>
```
Returns `n` values of `x` associated with the `n` smallest of all input values of `y`
in ascending order of `y`.
### sum
```
sum(x) -> [same as input]
```
Returns the sum of all input values.
## Bitwise aggregate functions
### bitwise\_and\_agg
```
bitwise_and_agg(x) -> bigint
```
Returns the bitwise AND of all input values in 2's complement representation.
### bitwise\_or\_agg
```
bitwise_or_agg(x) -> bigint
Returns the bitwise OR of all input values in 2's complement representation.
```
## Map aggregate functions
### histogram
```
histogram(x) -> map
```
Returns a map containing the count of the number of times each input value occurs.
### map\_agg
```
map_agg(key, value) -> map
```
Returns a map created from the input `key` / `value` pairs.
### map\_union
```
map_union(x(K,V)) -> map
```
Returns the union of all the input maps. If a key is found in multiple
input maps, that key's value in the resulting map comes from an arbitrary input map.
For example, take the following histogram function that creates multiple maps from the Iris dataset:
```sql theme={null}
SELECT histogram(floor(petal_length_cm)) petal_data
FROM memory.default.iris
GROUP BY species;
petal_data
-- {4.0=6, 5.0=33, 6.0=11}
-- {4.0=37, 5.0=2, 3.0=11}
-- {1.0=50}
```
You can combine these maps using `map_union`:
```sql theme={null}
SELECT map_union(petal_data) petal_data_union
FROM (
SELECT histogram(floor(petal_length_cm)) petal_data
FROM memory.default.iris
GROUP BY species
);
petal_data_union
--{4.0=6, 5.0=2, 6.0=11, 1.0=50, 3.0=11}
```
### multimap\_agg
```
multimap_agg(key, value) -> map
```
Returns a multimap created from the input `key` / `value` pairs.
Each key can be associated with multiple values.
## Approximate aggregate functions
### approx\_distinct
```
approx_distinct(x) -> bigint
```
Returns the approximate number of distinct input values.
This function provides an approximation of `count(DISTINCT x)`.
Zero is returned if all input values are null.
This function should produce a standard error of 2.3%, which is the
standard deviation of the (approximately normal) error distribution over
all possible sets. It does not guarantee an upper bound on the error for
any specific input set.
```
approx_distinct(x, e) -> bigint
```
Returns the approximate number of distinct input values.
This function provides an approximation of `count(DISTINCT x)`.
Zero is returned if all input values are null.
This function should produce a standard error of no more than `e`, which
is the standard deviation of the (approximately normal) error distribution
over all possible sets. It does not guarantee an upper bound on the error
for any specific input set. The current implementation of this function
requires that `e` be in the range of `[0.0040625, 0.26000]`.
### approx\_most\_frequent
```
approx_most_frequent(buckets, value, capacity) -> map<[same as value], bigint>
```
Computes the top frequent values up to `buckets` elements approximately.
Approximate estimation of the function enables us to pick up the frequent
values with less memory. Larger `capacity` improves the accuracy of
underlying algorithm with sacrificing the memory capacity. The returned
value is a map containing the top elements with corresponding estimated
frequency.
The error of the function depends on the permutation of the values and its
cardinality. We can set the capacity same as the cardinality of the
underlying data to achieve the least error.
`buckets` and `capacity` must be `bigint`. `value` can be numeric
or string type.
The function uses the stream summary data structure proposed in the paper
[Efficient Computation of Frequent and Top-k Elements in Data Streams](https://www.cse.ust.hk/~raywong/comp5331/References/EfficientComputationOfFrequentAndTop-kElementsInDataStreams.pdf)
by A. Metwalley, D. Agrawl and A. Abbadi.
### approx\_percentile
```
approx_percentile(x, percentage) -> [same as x]
```
Returns the approximate percentile for all input values of `x` at the
given `percentage`. The value of `percentage` must be between zero and
one and must be constant for all input rows.
```
approx_percentile(x, percentages) -> array<[same as x]>
```
Returns the approximate percentile for all input values of `x` at each of
the specified percentages. Each element of the `percentages` array must be
between zero and one, and the array must be constant for all input rows.
```
approx_percentile(x, w, percentage) -> [same as x]
```
Returns the approximate weighed percentile for all input values of `x`
using the per-item weight `w` at the percentage `percentage`. Weights must be
greater or equal to 1. Integer-value weights can be thought of as a replication
count for the value `x` in the percentile set. The value of `percentage` must be
between zero and one and must be constant for all input rows.
```
approx_percentile(x, w, percentages) -> array<[same as x]>
```
Returns the approximate weighed percentile for all input values of `x`
using the per-item weight `w` at each of the given percentages specified
in the array. Weights must be greater or equal to 1. Integer-value weights can
be thought of as a replication count for the value `x` in the percentile
set. Each element of the `percentages` array must be between zero and one, and the array
must be constant for all input rows.
### approx\_set
```
approx_set(x) -> HyperLogLog
```
See [HyperLogLog Functions](/sql/functions/hyperloglog).
### merge
```
merge(x) -> HyperLogLog
```
See [HyperLogLog Functions](/sql/functions/hyperloglog).
```
merge(qdigest(T)) -> qdigest(T)
```
See [Quantile digest functions](/sql/functions/qdigest).
```
merge(tdigest) -> tdigest
```
### numeric\_histogram
```
numeric_histogram(buckets, value) -> map
```
Computes an approximate histogram with up to `buckets` number of buckets
for all `value`s. This function is equivalent to the variant of
`numeric_histogram` that takes a `weight`, with a per-item weight of `1`.
```
numeric_histogram(buckets, value, weight) -> map
```
Computes an approximate histogram with up to `buckets` number of buckets
for all `value`s with a per-item weight of `weight`. The algorithm
is based loosely on:
```
Yael Ben-Haim and Elad Tom-Tov, "A streaming parallel decision tree algorithm",
J. Machine Learning Research 11 (2010), pp. 849--872.
```
`buckets` must be a `bigint`. `value` and `weight` must be numeric.
### qdigest\_agg
```
qdigest_agg(x) -> qdigest([same as x])
```
```
qdigest_agg(x, w) -> qdigest([same as x])
```
```
qdigest_agg(x, w, accuracy) -> qdigest([same as x])
```
### tdigest\_agg
```
tdigest_agg(x) -> tdigest
```
```
tdigest_agg(x, w) -> tdigest
```
## Statistical aggregate functions
### corr
```
corr(y, x) -> double
```
Returns correlation coefficient of input values.
### covar\_pop
```
covar_pop(y, x) -> double
```
Returns the population covariance of input values.
### covar\_samp
```
covar_samp(y, x) -> double
```
Returns the sample covariance of input values.
### kurtosis
```
kurtosis(x) -> double
```
Returns the excess kurtosis of all input values. Unbiased estimate using
the following expression:
```
kurtosis(x) = n(n+1)/((n-1)(n-2)(n-3))sum[(x_i-mean)^4]/stddev(x)^4-3(n-1)^2/((n-2)(n-3))
```
### regr\_intercept
```
regr_intercept(y, x) -> double
```
Returns linear regression intercept of input values. `y` is the dependent
value. `x` is the independent value.
### regr\_slope
```
regr_slope(y, x) -> double
```
Returns linear regression slope of input values. `y` is the dependent
value. `x` is the independent value.
### skewness
```
skewness(x) -> double
```
Returns the Fisher's moment coefficient of [skewness](https://wikipedia.org/wiki/Skewness) of all input values.
### stddev
```
stddev(x) -> double
```
This is an alias for `stddev_samp`.
### stddev\_pop
```
stddev_pop(x) -> double
```
Returns the population standard deviation of all input values.
### stddev\_samp
```
stddev_samp(x) -> double
```
Returns the sample standard deviation of all input values.
### variance
```
variance(x) -> double
```
This is an alias for `var_samp`.
### var\_pop
```
var_pop(x) -> double
```
Returns the population variance of all input values.
### var\_samp
```
var_samp(x) -> double
```
Returns the sample variance of all input values.
## Lambda aggregate functions
### reduce\_agg
```
reduce_agg(inputValue T, initialState S, inputFunction(S, T, S), combineFunction(S, S, S)) -> S
```
Reduces all input values into a single value. `inputFunction` will be invoked
for each non-null input value. In addition to taking the input value, `inputFunction`
takes the current state, initially `initialState`, and returns the new state.
`combineFunction` will be invoked to combine two states into a new state.
The final state is returned:
```sql theme={null}
SELECT id, reduce_agg(value, 0, (a, b) -> a + b, (a, b) -> a + b)
FROM (
VALUES
(1, 3),
(1, 4),
(1, 5),
(2, 6),
(2, 7)
) AS t(id, value)
GROUP BY id;
-- (1, 12)
-- (2, 13)
SELECT id, reduce_agg(value, 1, (a, b) -> a * b, (a, b) -> a * b)
FROM (
VALUES
(1, 3),
(1, 4),
(1, 5),
(2, 6),
(2, 7)
) AS t(id, value)
GROUP BY id;
-- (1, 60)
-- (2, 42)
```
The state type must be a boolean, integer, floating-point, char, varchar or date/time/interval.
# Array functions and operators
Source: https://docs.upsolve.ai/sql/functions/array
## Subscript operator: \[]
The `[]` operator is used to access an element of an array and is indexed starting from one:
```sql theme={null}
SELECT my_array[1] AS first_element
```
## Concatenation operator: ||
The `||` operator is used to concatenate an array with an array or an element of the same type:
```sql theme={null}
SELECT ARRAY[1] || ARRAY[2];
-- [1, 2]
SELECT ARRAY[1] || 2;
-- [1, 2]
SELECT 2 || ARRAY[1];
-- [2, 1]
```
## Array functions
### all\_match
```
all_match(array(T), function(T,boolean)) -> boolean
```
Returns whether all elements of an array match the given predicate. Returns `true` if all the elements
match the predicate (a special case is when the array is empty); `false` if one or more elements don't
match; `NULL` if the predicate function returns `NULL` for one or more elements and `true` for all
other elements.
### any\_match
```
any_match(array(T), function(T,boolean)) -> boolean
```
Returns whether any elements of an array match the given predicate. Returns `true` if one or more
elements match the predicate; `false` if none of the elements matches (a special case is when the
array is empty); `NULL` if the predicate function returns `NULL` for one or more elements and `false`
for all other elements.
### array\_distinct
```
array_distinct(x) -> array
```
Remove duplicate values from the array `x`.
### array\_intersect
```
array_intersect(x, y) -> array
```
Returns an array of the elements in the intersection of `x` and `y`, without duplicates.
### array\_union
```
array_union(x, y) -> array
```
Returns an array of the elements in the union of `x` and `y`, without duplicates.
### array\_except
```
array_except(x, y) -> array
```
Returns an array of elements in `x` but not in `y`, without duplicates.
### array\_histogram
```
array_histogram(x) -> map
```
Returns a map where the keys are the unique elements in the input array
`x` and the values are the number of times that each element appears in
`x`. Null values are ignored.
```sql theme={null}
SELECT array_histogram(ARRAY[42, 7, 42, NULL]);
-- {42=2, 7=1}
```
Returns an empty map if the input array has no non-null elements.
```sql theme={null}
SELECT array_histogram(ARRAY[NULL, NULL]);
-- {}
```
### array\_join
```
array_join(x, delimiter) -> varchar
```
Concatenates the elements of the given array using the delimiter.
Null elements are omitted in the result.
```
array_join(x, delimiter, null_replacement) -> varchar
```
Concatenates the elements of the given array using the delimiter and an optional string to replace nulls.
### array\_max
```
array_max(x) -> x
```
Returns the maximum value of input array.
### array\_min
```
array_min(x) -> x
```
Returns the minimum value of input array.
### array\_position
```
array_position(x, element) -> bigint
```
Returns the position of the first occurrence of the `element` in array `x` (or 0 if not found).
### array\_remove
```
array_remove(x, element) -> array
```
Remove all elements that equal `element` from array `x`.
### array\_sort
```
array_sort(x) -> array
```
Sorts and returns the array `x`. The elements of `x` must be orderable.
Null elements will be placed at the end of the returned array.
```
array_sort(array(T), function(T,T,int)) -> array(T)
```
Sorts and returns the `array` based on the given comparator `function`.
The comparator will take two nullable arguments representing two nullable
elements of the `array`. It returns -1, 0, or 1 as the first nullable
element is less than, equal to, or greater than the second nullable element.
If the comparator function returns other values (including `NULL`), the
query will fail and raise an error.
```sql theme={null}
SELECT array_sort(ARRAY[3, 2, 5, 1, 2],
(x, y) -> IF(x < y, 1, IF(x = y, 0, -1)));
-- [5, 3, 2, 2, 1]
SELECT array_sort(ARRAY['bc', 'ab', 'dc'],
(x, y) -> IF(x < y, 1, IF(x = y, 0, -1)));
-- ['dc', 'bc', 'ab']
SELECT array_sort(ARRAY[3, 2, null, 5, null, 1, 2],
-- sort null first with descending order
(x, y) -> CASE WHEN x IS NULL THEN -1
WHEN y IS NULL THEN 1
WHEN x < y THEN 1
WHEN x = y THEN 0
ELSE -1 END);
-- [null, null, 5, 3, 2, 2, 1]
SELECT array_sort(ARRAY[3, 2, null, 5, null, 1, 2],
-- sort null last with descending order
(x, y) -> CASE WHEN x IS NULL THEN 1
WHEN y IS NULL THEN -1
WHEN x < y THEN 1
WHEN x = y THEN 0
ELSE -1 END);
-- [5, 3, 2, 2, 1, null, null]
SELECT array_sort(ARRAY['a', 'abcd', 'abc'],
-- sort by string length
(x, y) -> IF(length(x) < length(y), -1,
IF(length(x) = length(y), 0, 1)));
-- ['a', 'abc', 'abcd']
SELECT array_sort(ARRAY[ARRAY[2, 3, 1], ARRAY[4, 2, 1, 4], ARRAY[1, 2]],
-- sort by array length
(x, y) -> IF(cardinality(x) < cardinality(y), -1,
IF(cardinality(x) = cardinality(y), 0, 1)));
-- [[1, 2], [2, 3, 1], [4, 2, 1, 4]]
```
### arrays\_overlap
```
arrays_overlap(x, y) -> boolean
```
Tests if arrays `x` and `y` have any non-null elements in common.
Returns null if there are no non-null elements in common but either array contains null.
### cardinality
```
cardinality(x) -> bigint
```
Returns the cardinality (size) of the array `x`.
### concat
```
concat(array1, array2, ..., arrayN) -> array
```
Concatenates the arrays `array1`, `array2`, `...`, `arrayN`.
This function provides the same functionality as the SQL-standard concatenation operator (`||`).
### combinations
```
combinations(array(T), n) -> array(array(T))
```
Returns n-element sub-groups of input array. If the input array has no duplicates,
`combinations` returns n-element subsets.
```sql theme={null}
SELECT combinations(ARRAY['foo', 'bar', 'baz'], 2);
-- [['foo', 'bar'], ['foo', 'baz'], ['bar', 'baz']]
SELECT combinations(ARRAY[1, 2, 3], 2);
-- [[1, 2], [1, 3], [2, 3]]
SELECT combinations(ARRAY[1, 2, 2], 2);
-- [[1, 2], [1, 2], [2, 2]]
```
Order of sub-groups is deterministic but unspecified. Order of elements within
a sub-group deterministic but unspecified. `n` must be not be greater than 5,
and the total size of sub-groups generated must be smaller than 100,000.
### contains
```
contains(x, element) -> boolean
```
Returns true if the array `x` contains the `element`.
### contains\_sequence
```
contains_sequence(x, seq) -> boolean
```
Return true if array `x` contains all of array `seq` as a subsequence (all values in the same consecutive order).
### element\_at
```
element_at(array(E), index) -> E
```
Returns element of `array` at given `index`.
If `index` > 0, this function provides the same functionality as the SQL-standard subscript operator (`[]`),
except that the function returns `NULL` when accessing an `index` larger than array length, whereas
the subscript operator would fail in such a case.
If `index` \< 0, `element_at` accesses elements from the last to the first.
### filter
```
filter(array(T), function(T,boolean)) -> array(T)
```
Constructs an array from those elements of `array` for which `function` returns true:
```sql theme={null}
SELECT filter(ARRAY[], x -> true);
-- []
SELECT filter(ARRAY[5, -6, NULL, 7], x -> x > 0);
-- [5, 7]
SELECT filter(ARRAY[5, NULL, 7, NULL], x -> x IS NOT NULL);
-- [5, 7]
```
### flatten
```
flatten(x) -> array
```
Flattens an `array(array(T))` to an `array(T)` by concatenating the contained arrays.
### ngrams
```
ngrams(array(T), n) -> array(array(T))
```
Returns `n`-grams (sub-sequences of adjacent `n` elements) for the `array`.
The order of the `n`-grams in the result is unspecified.
```sql theme={null}
SELECT ngrams(ARRAY['foo', 'bar', 'baz', 'foo'], 2);
-- [['foo', 'bar'], ['bar', 'baz'], ['baz', 'foo']]
SELECT ngrams(ARRAY['foo', 'bar', 'baz', 'foo'], 3);
-- [['foo', 'bar', 'baz'], ['bar', 'baz', 'foo']]
SELECT ngrams(ARRAY['foo', 'bar', 'baz', 'foo'], 4);
-- [['foo', 'bar', 'baz', 'foo']]
SELECT ngrams(ARRAY['foo', 'bar', 'baz', 'foo'], 5);
-- [['foo', 'bar', 'baz', 'foo']]
SELECT ngrams(ARRAY[1, 2, 3, 4], 2);
-- [[1, 2], [2, 3], [3, 4]]
```
### none\_match
```
none_match(array(T), function(T,boolean)) -> boolean
```
Returns whether no elements of an array match the given predicate. Returns `true` if none of the elements
matches the predicate (a special case is when the array is empty); `false` if one or more elements match;
`NULL` if the predicate function returns `NULL` for one or more elements and `false` for all other elements.
### reduce
```
reduce(array(T), initialState S, inputFunction(S,T,S), outputFunction(S,R)) -> R
```
Returns a single value reduced from `array`. `inputFunction` will
be invoked for each element in `array` in order. In addition to taking
the element, `inputFunction` takes the current state, initially
`initialState`, and returns the new state. `outputFunction` will be
invoked to turn the final state into the result value. It may be the
identity function (`i -> i`).
```sql theme={null}
SELECT reduce(ARRAY[], 0,
(s, x) -> s + x,
s -> s);
-- 0
SELECT reduce(ARRAY[5, 20, 50], 0,
(s, x) -> s + x,
s -> s);
-- 75
SELECT reduce(ARRAY[5, 20, NULL, 50], 0,
(s, x) -> s + x,
s -> s);
-- NULL
SELECT reduce(ARRAY[5, 20, NULL, 50], 0,
(s, x) -> s + coalesce(x, 0),
s -> s);
-- 75
SELECT reduce(ARRAY[5, 20, NULL, 50], 0,
(s, x) -> IF(x IS NULL, s, s + x),
s -> s);
-- 75
SELECT reduce(ARRAY[2147483647, 1], BIGINT '0',
(s, x) -> s + x,
s -> s);
-- 2147483648
-- calculates arithmetic average
SELECT reduce(ARRAY[5, 6, 10, 20],
CAST(ROW(0.0, 0) AS ROW(sum DOUBLE, count INTEGER)),
(s, x) -> CAST(ROW(x + s.sum, s.count + 1) AS
ROW(sum DOUBLE, count INTEGER)),
s -> IF(s.count = 0, NULL, s.sum / s.count));
-- 10.25
```
### repeat
```
repeat(element, count) -> array
```
Repeat `element` for `count` times.
### reverse
```
reverse(x) -> array
```
Returns an array which has the reversed order of array `x`.
### sequence
```
sequence(start, stop) -> array(bigint)
```
Generate a sequence of integers from `start` to `stop`, incrementing
by `1` if `start` is less than or equal to `stop`, otherwise `-1`.
```
sequence(start, stop, step) -> array(bigint)
```
Generate a sequence of integers from `start` to `stop`, incrementing by `step`.
```
sequence(start, stop) -> array(date)
```
Generate a sequence of dates from `start` date to `stop` date, incrementing
by `1` day if `start` date is less than or equal to `stop` date, otherwise `-1` day.
```
sequence(start, stop, step) -> array(date)
```
Generate a sequence of dates from `start` to `stop`, incrementing by `step`.
The type of `step` can be either `INTERVAL DAY TO SECOND` or `INTERVAL YEAR TO MONTH`.
```
sequence(start, stop, step) -> array(timestamp)
```
Generate a sequence of timestamps from `start` to `stop`, incrementing by `step`.
The type of `step` can be either `INTERVAL DAY TO SECOND` or `INTERVAL YEAR TO MONTH`.
### shuffle
```
shuffle(x) -> array
```
Generate a random permutation of the given array `x`.
### slice
```
slice(x, start, length) -> array
```
Subsets array `x` starting from index `start` (or starting from the end
if `start` is negative) with a length of `length`.
### trim\_array
```
trim_array(x, n) -> array
```
Remove `n` elements from the end of array:
```sql theme={null}
SELECT trim_array(ARRAY[1, 2, 3, 4], 1);
-- [1, 2, 3]
SELECT trim_array(ARRAY[1, 2, 3, 4], 2);
-- [1, 2]
```
### transform
```
transform(array(T), function(T,U)) -> array(U)
```
Returns an array that is the result of applying `function` to each element of `array`:
```sql theme={null}
SELECT transform(ARRAY[], x -> x + 1);
-- []
SELECT transform(ARRAY[5, 6], x -> x + 1);
-- [6, 7]
SELECT transform(ARRAY[5, NULL, 6], x -> coalesce(x, 0) + 1);
-- [6, 1, 7]
SELECT transform(ARRAY['x', 'abc', 'z'], x -> x || '0');
-- ['x0', 'abc0', 'z0']
SELECT transform(ARRAY[ARRAY[1, NULL, 2], ARRAY[3, NULL]],
a -> filter(a, x -> x IS NOT NULL));
-- [[1, 2], [3]]
```
### zip
```
zip(array1, array2[, ...]) -> array(row)
```
Merges the given arrays, element-wise, into a single array of rows. The M-th element of
the N-th argument will be the N-th field of the M-th output element.
If the arguments have an uneven length, missing values are filled with `NULL`.
```sql theme={null}
SELECT zip(ARRAY[1, 2], ARRAY['1b', null, '3b']);
-- [ROW(1, '1b'), ROW(2, null), ROW(null, '3b')]
```
### zip\_with
```
zip_with(array(T), array(U), function(T,U,R)) -> array(R)
```
Merges the two given arrays, element-wise, into a single array using `function`.
If one array is shorter, nulls are appended at the end to match the length of the
longer array, before applying `function`.
```sql theme={null}
SELECT zip_with(ARRAY[1, 3, 5], ARRAY['a', 'b', 'c'],
(x, y) -> (y, x));
-- [ROW('a', 1), ROW('b', 3), ROW('c', 5)]
SELECT zip_with(ARRAY[1, 2], ARRAY[3, 4],
(x, y) -> x + y);
-- [4, 6]
SELECT zip_with(ARRAY['a', 'b', 'c'], ARRAY['d', 'e', 'f'],
(x, y) -> concat(x, y));
-- ['ad', 'be', 'cf']
SELECT zip_with(ARRAY['a'], ARRAY['d', null, 'f'],
(x, y) -> coalesce(x, y));
-- ['a', null, 'f']
```
# Binary functions and operators
Source: https://docs.upsolve.ai/sql/functions/binary
## Binary operators
The `||` operator performs concatenation.
## Binary functions
### concat
```
concat(binary1, ..., binaryN) -> varbinary
```
Returns the concatenation of `binary1`, `binary2`, `...`, `binaryN`.
This function provides the same functionality as the
SQL-standard concatenation operator (`||`).
### length
```
length(binary) -> bigint
```
Returns the length of `binary` in bytes.
### lpad
```
lpad(binary, size, padbinary) -> varbinary
```
Left pads `binary` to `size` bytes with `padbinary`.
If `size` is less than the length of `binary`, the result is
truncated to `size` characters. `size` must not be negative
and `padbinary` must be non-empty.
### rpad
```
rpad(binary, size, padbinary) -> varbinary
```
Right pads `binary` to `size` bytes with `padbinary`.
If `size` is less than the length of `binary`, the result is
truncated to `size` characters. `size` must not be negative
and `padbinary` must be non-empty.
### substr
```
substr(binary, start) -> varbinary
```
Returns the rest of `binary` from the starting position `start`,
measured in bytes. Positions start with `1`. A negative starting position
is interpreted as being relative to the end of the string.
```
substr(binary, start, length) -> varbinary
```
Returns a substring from `binary` of length `length` from the starting
position `start`, measured in bytes. Positions start with `1`. A
negative starting position is interpreted as being relative to the end of
the string.
### reverse
```
reverse(binary) -> varbinary
```
Returns `binary` with the bytes in reverse order.
## Base64 encoding functions
The Base64 functions implement the encoding specified in RFC 4648.
### from\_base64
```
from_base64(string) -> varbinary
```
Decodes binary data from the base64 encoded `string`.
### to\_base64
```
to_base64(binary) -> varchar
```
Encodes `binary` into a base64 string representation.
### from\_base64url
```
from_base64url(string) -> varbinary
```
Decodes binary data from the base64 encoded `string` using the URL safe alphabet.
### to\_base64url
```
to_base64url(binary) -> varchar
```
Encodes `binary` into a base64 string representation using the URL safe alphabet.
### from\_base32
```
from_base32(string) -> varbinary
```
Decodes binary data from the base32 encoded `string`.
### to\_base32
```
to_base32(binary) -> varchar
```
Encodes `binary` into a base32 string representation.
## Hex encoding functions
### from\_hex
```
from_hex(string) -> varbinary
```
Decodes binary data from the hex encoded `string`.
### to\_hex
```
to_hex(binary) -> varchar
```
Encodes `binary` into a hex string representation.
## Integer encoding functions
### from\_big\_endian\_32
```
from_big_endian_32(binary) -> integer
```
Decodes the 32-bit two's complement big-endian `binary`.
The input must be exactly 4 bytes.
### to\_big\_endian\_32
```
to_big_endian_32(integer) -> varbinary
```
Encodes `integer` into a 32-bit two's complement big-endian format.
### from\_big\_endian\_64
```
from_big_endian_64(binary) -> bigint
```
Decodes the 64-bit two's complement big-endian `binary`.
The input must be exactly 8 bytes.
### to\_big\_endian\_64
```
to_big_endian_64(bigint) -> varbinary
```
Encodes `bigint` into a 64-bit two's complement big-endian format.
## Floating-point encoding functions
### from\_ieee754\_32
```
from_ieee754_32(binary) -> real
```
Decodes the 32-bit big-endian `binary` in IEEE 754 single-precision floating-point format.
The input must be exactly 4 bytes.
### to\_ieee754\_32
```
to_ieee754_32(real) -> varbinary
```
Encodes `real` into a 32-bit big-endian binary according to IEEE 754 single-precision floating-point format.
### from\_ieee754\_64
```
from_ieee754_64(binary) -> double
```
Decodes the 64-bit big-endian `binary` in IEEE 754 double-precision floating-point format.
The input must be exactly 8 bytes.
### to\_ieee754\_64
```
to_ieee754_64(double) -> varbinary
```
Encodes `double` into a 64-bit big-endian binary according to IEEE 754 double-precision floating-point format.
## Hashing functions
### crc32
```
crc32(binary) -> bigint
```
Computes the CRC-32 of `binary`. For general purpose hashing, use
`xxhash64`, as it is much faster and produces a better quality hash.
### md5
```
md5(binary) -> varbinary
```
Computes the MD5 hash of `binary`.
### sha1
```
sha1(binary) -> varbinary
```
Computes the SHA1 hash of `binary`.
### sha256
```
sha256(binary) -> varbinary
```
Computes the SHA256 hash of `binary`.
### sha512
```
sha512(binary) -> varbinary
```
Computes the SHA512 hash of `binary`.
### spooky\_hash\_v2\_32
```
spooky_hash_v2_32(binary) -> varbinary
```
Computes the 32-bit SpookyHashV2 hash of `binary`.
### spooky\_hash\_v2\_64
```
spooky_hash_v2_64(binary) -> varbinary
```
Computes the 64-bit SpookyHashV2 hash of `binary`.
### xxhash64
```
xxhash64(binary) -> varbinary
```
Computes the xxHash64 hash of `binary`.
### murmur3
```
murmur3(binary) -> varbinary
```
Computes the 128-bit [MurmurHash3](https://wikipedia.org/wiki/MurmurHash)
hash of `binary`.
```sql theme={null}
> SELECT murmur3(from_base64('aaaaaa'));
> -- ba 58 55 63 55 69 b4 2f 49 20 37 2c a0 e3 96 ef
```
## HMAC functions
### hmac\_md5
```
hmac_md5(binary, key) -> varbinary
```
Computes HMAC with MD5 of `binary` with the given `key`.
### hmac\_sha1
```
hmac_sha1(binary, key) -> varbinary
```
Computes HMAC with SHA1 of `binary` with the given `key`.
### hmac\_sha256
```
hmac_sha256(binary, key) -> varbinary
```
Computes HMAC with SHA256 of `binary` with the given `key`.
### hmac\_sha512
```
hmac_sha512(binary, key) -> varbinary
```
Computes HMAC with SHA512 of `binary` with the given `key`.
# Bitwise functions
Source: https://docs.upsolve.ai/sql/functions/bitwise
## Bitwise functions
### bit\_count
```
bit_count(x, bits) -> bigint
```
Count the number of bits set in `x` (treated as `bits`-bit signed
integer) in 2's complement representation:
```sql theme={null}
SELECT bit_count(9, 64); -- 2
SELECT bit_count(9, 8); -- 2
SELECT bit_count(-7, 64); -- 62
SELECT bit_count(-7, 8); -- 6
```
### bitwise\_and
```
bitwise_and(x, y) -> bigint
```
Returns the bitwise AND of `x` and `y` in 2's complement representation.
Bitwise AND of `19` (binary: `10011`) and `25` (binary: `11001`) results in
`17` (binary: `10001`):
```sql theme={null}
SELECT bitwise_and(19,25); -- 17
```
### bitwise\_not
```
bitwise_not(x) -> bigint
```
Returns the bitwise NOT of `x` in 2's complement representation
(`NOT x = -x - 1`):
```sql theme={null}
SELECT bitwise_not(-12); -- 11
SELECT bitwise_not(19); -- -20
SELECT bitwise_not(25); -- -26
```
### bitwise\_or
```
bitwise_or(x, y) -> bigint
```
Returns the bitwise OR of `x` and `y` in 2's complement representation.
Bitwise OR of `19` (binary: `10011`) and `25` (binary: `11001`) results in
`27` (binary: `11011`):
```sql theme={null}
SELECT bitwise_or(19,25); -- 27
```
### bitwise\_xor
```
bitwise_xor(x, y) -> bigint
```
Returns the bitwise XOR of `x` and `y` in 2's complement representation.
Bitwise XOR of `19` (binary: `10011`) and `25` (binary: `11001`) results in
`10` (binary: `01010`):
```sql theme={null}
SELECT bitwise_xor(19,25); -- 10
```
### bitwise\_left\_shift
```
bitwise_left_shift(value, shift) -> [same as value]
```
Returns the left shifted value of `value`.
Shifting `1` (binary: `001`) by two bits results in `4` (binary: `00100`):
```sql theme={null}
SELECT bitwise_left_shift(1, 2); -- 4
```
Shifting `5` (binary: `0101`) by two bits results in `20` (binary: `010100`):
```sql theme={null}
SELECT bitwise_left_shift(5, 2); -- 20
```
Shifting a `value` by `0` always results in the original `value`:
```sql theme={null}
SELECT bitwise_left_shift(20, 0); -- 20
SELECT bitwise_left_shift(42, 0); -- 42
```
Shifting `0` by a `shift` always results in `0`:
```sql theme={null}
SELECT bitwise_left_shift(0, 1); -- 0
SELECT bitwise_left_shift(0, 2); -- 0
```
### bitwise\_right\_shift
```
bitwise_right_shift(value, shift) -> [same as value]
```
Returns the logical right shifted value of `value`.
Shifting `8` (binary: `1000`) by three bits results in `1` (binary: `001`):
```sql theme={null}
SELECT bitwise_right_shift(8, 3); -- 1
```
Shifting `9` (binary: `1001`) by one bit results in `4` (binary: `100`):
```sql theme={null}
SELECT bitwise_right_shift(9, 1); -- 4
```
Shifting a `value` by `0` always results in the original `value`:
```sql theme={null}
SELECT bitwise_right_shift(20, 0); -- 20
SELECT bitwise_right_shift(42, 0); -- 42
```
Shifting a `value` by `64` or more bits results in `0`:
```sql theme={null}
SELECT bitwise_right_shift( 12, 64); -- 0
SELECT bitwise_right_shift(-45, 64); -- 0
```
Shifting `0` by a `shift` always results in `0`:
```sql theme={null}
SELECT bitwise_right_shift(0, 1); -- 0
SELECT bitwise_right_shift(0, 2); -- 0
```
### bitwise\_right\_shift\_arithmetic
```
bitwise_right_shift_arithmetic(value, shift) -> [same as value]
```
Returns the arithmetic right shifted value of `value`.
Returns the same values as [bitwise\_right\_shift](#bitwise_right_shift) when shifting by less than
`64` bits. Shifting by `64` or more bits results in `0` for a positive and
`-1` for a negative `value`:
```sql theme={null}
SELECT bitwise_right_shift_arithmetic( 12, 64); -- 0
SELECT bitwise_right_shift_arithmetic(-45, 64); -- -1
```
See also [bitwise\_and\_agg](/sql/functions/aggregate#bitwise-and-agg) and [bitwise\_or\_agg](/sql/functions/aggregate#bitwise-or-agg).
# Comparison functions and operators
Source: https://docs.upsolve.ai/sql/functions/comparison
## Comparison operators
| Operator | Description |
| -------- | ------------------------ |
| \< | Less than |
| > | Greater than |
| \<= | Less than or equal to |
| >= | Greater than or equal to |
| = | Equal |
| \<> | Not equal |
| != | Not equal |
## Range operator: BETWEEN
The `BETWEEN` operator tests if a value is within a specified range.
It uses the syntax `value BETWEEN min AND max`:
```sql theme={null}
SELECT 3 BETWEEN 2 AND 6;
```
The statement shown above is equivalent to the following statement:
```sql theme={null}
SELECT 3 >= 2 AND 3 <= 6;
```
To test if a value does not fall within the specified range
use `NOT BETWEEN`:
```sql theme={null}
SELECT 3 NOT BETWEEN 2 AND 6;
```
The statement shown above is equivalent to the following statement:
```sql theme={null}
SELECT 3 < 2 OR 3 > 6;
```
A `NULL` in a `BETWEEN` or `NOT BETWEEN` statement is evaluated
using the standard `NULL` evaluation rules applied to the equivalent
expression above:
```sql theme={null}
SELECT NULL BETWEEN 2 AND 4; -- null
SELECT 2 BETWEEN NULL AND 6; -- null
SELECT 2 BETWEEN 3 AND NULL; -- false
SELECT 8 BETWEEN NULL AND 6; -- false
```
The `BETWEEN` and `NOT BETWEEN` operators can also be used to
evaluate any orderable type. For example, a `VARCHAR`:
```sql theme={null}
SELECT 'Paul' BETWEEN 'John' AND 'Ringo'; -- true
```
Note that the value, min, and max parameters to `BETWEEN` and `NOT
BETWEEN` must be the same type. For example, Trino will produce an
error if you ask it if John is between 2.3 and 35.2.
## IS NULL and IS NOT NULL
The `IS NULL` and `IS NOT NULL` operators test whether a value
is null (undefined). Both operators work for all data types.
Using `NULL` with `IS NULL` evaluates to true:
```sql theme={null}
select NULL IS NULL; -- true
```
But any other constant does not:
```sql theme={null}
SELECT 3.0 IS NULL; -- false
```
## IS DISTINCT FROM and IS NOT DISTINCT FROM
In SQL a `NULL` value signifies an unknown value, so any comparison
involving a `NULL` will produce `NULL`. The `IS DISTINCT FROM`
and `IS NOT DISTINCT FROM` operators treat `NULL` as a known value
and both operators guarantee either a true or false outcome even in
the presence of `NULL` input:
```sql theme={null}
SELECT NULL IS DISTINCT FROM NULL; -- false
SELECT NULL IS NOT DISTINCT FROM NULL; -- true
```
In the example shown above, a `NULL` value is not considered
distinct from `NULL`. When you are comparing values which may
include `NULL` use these operators to guarantee either a `TRUE` or
`FALSE` result.
The following truth table demonstrate the handling of `NULL` in
`IS DISTINCT FROM` and `IS NOT DISTINCT FROM`:
| a | b | a = b | a \<> b | a DISTINCT b | a NOT DISTINCT b |
| ------ | ------ | ------- | ------- | ------------ | ---------------- |
| `1` | `1` | `TRUE` | `FALSE` | `FALSE` | `TRUE` |
| `1` | `2` | `FALSE` | `TRUE` | `TRUE` | `FALSE` |
| `1` | `NULL` | `NULL` | `NULL` | `TRUE` | `FALSE` |
| `NULL` | `NULL` | `NULL` | `NULL` | `FALSE` | `TRUE` |
## GREATEST and LEAST
These functions are not in the SQL standard, but are a common extension.
Like most other functions in Trino, they return null if any argument is
null. Note that in some other databases, such as PostgreSQL, they only
return null if all arguments are null.
The following types are supported:
`DOUBLE`,
`BIGINT`,
`VARCHAR`,
`TIMESTAMP`,
`TIMESTAMP WITH TIME ZONE`,
`DATE`
```
greatest(value1, value2, ..., valueN) -> [same as input]
```
Returns the largest of the provided values.
```
least(value1, value2, ..., valueN) -> [same as input]
```
Returns the smallest of the provided values.
## Quantified comparison predicates: ALL, ANY and SOME
The `ALL`, `ANY` and `SOME` quantifiers can be used together with comparison operators in the
following way:
```text theme={null}
expression operator quantifier ( subquery )
```
For example:
```sql theme={null}
SELECT 'hello' = ANY (VALUES 'hello', 'world'); -- true
SELECT 21 < ALL (VALUES 19, 20, 21); -- false
SELECT 42 >= SOME (SELECT 41 UNION ALL SELECT 42 UNION ALL SELECT 43); -- true
```
Here are the meanings of some quantifier and comparison operator combinations:
| Expression | Meaning |
| ---------------- | ---------------------------------------------------------------------------------------------------- |
| `A = ALL (...)` | Evaluates to `true` when `A` is equal to all values. |
| `A <> ALL (...)` | Evaluates to `true` when `A` doesn't match any value. |
| `A < ALL (...)` | Evaluates to `true` when `A` is smaller than the smallest value. |
| `A = ANY (...)` | Evaluates to `true` when `A` is equal to any of the values. This form is equivalent to `A IN (...)`. |
| `A <> ANY (...)` | Evaluates to `true` when `A` doesn't match one or more values. |
| `A < ANY (...)` | Evaluates to `true` when `A` is smaller than the biggest value. |
`ANY` and `SOME` have the same meaning and can be used interchangeably.
## Pattern comparison: LIKE
The `LIKE` operator can be used to compare values with a pattern:
```sql theme={null}
... column [NOT] LIKE 'pattern' ESCAPE 'character';
```
Matching characters is case sensitive, and the pattern supports two symbols for
matching:
* `_` matches any single character
* `%` matches zero or more characters
Typically it is often used as a condition in `WHERE` statements. An example is
a query to find all continents starting with `E`, which returns `Europe`:
```sql theme={null}
SELECT * FROM (VALUES 'America', 'Asia', 'Africa', 'Europe', 'Australia', 'Antarctica') AS t (continent)
WHERE continent LIKE 'E%';
```
You can negate the result by adding `NOT`, and get all other continents, all
not starting with `E`:
```sql theme={null}
SELECT * FROM (VALUES 'America', 'Asia', 'Africa', 'Europe', 'Australia', 'Antarctica') AS t (continent)
WHERE continent NOT LIKE 'E%';
```
If you only have one specific character to match, you can use the `_` symbol
for each character. The following query uses two underscores and produces only
`Asia` as result:
```sql theme={null}
SELECT * FROM (VALUES 'America', 'Asia', 'Africa', 'Europe', 'Australia', 'Antarctica') AS t (continent)
WHERE continent LIKE 'A__A';
```
The wildcard characters `_` and `%` must be escaped to allow you to match
them as literals. This can be achieved by specifying the `ESCAPE` character to
use:
```sql theme={null}
SELECT 'South_America' LIKE 'South\_America' ESCAPE '\';
```
The above query returns `true` since the escaped underscore symbol matches. If
you need to match the used escape character as well, you can escape it.
If you want to match for the chosen escape character, you simply escape itself.
For example, you can use `\\` to match for `\`.
## Row comparison: IN
The `IN` operator can be used in a `WHERE` clause to compare column values with
a list of values. The list of values can be supplied by a subquery or directly
as static values in an array:
```sql theme={null}
... WHERE column [NOT] IN ('value1','value2');
... WHERE column [NOT] IN ( subquery );
```
Use the optional `NOT` keyword to negate the condition.
The following example shows a simple usage with a static array:
```sql theme={null}
SELECT * FROM region WHERE name IN ('AMERICA', 'EUROPE');
```
The values in the clause are used for multiple comparisons that are combined as
a logical `OR`. The preceding query is equivalent to the following query:
```sql theme={null}
SELECT * FROM region WHERE name = 'AMERICA' OR name = 'EUROPE';
```
You can negate the comparisons by adding `NOT`, and get all other regions
except the values in list:
```sql theme={null}
SELECT * FROM region WHERE name NOT IN ('AMERICA', 'EUROPE');
```
When using a subquery to determine the values to use in the comparison, the
subquery must return a single column and one or more rows.
```sql theme={null}
SELECT name
FROM nation
WHERE regionkey IN (
SELECT starts_with(regionkey,"A") AS regionkey
FROM region
);
```
# Conditional Expressions
Source: https://docs.upsolve.ai/sql/functions/conditional
## CASE
The standard SQL `CASE` expression has two forms.
The "simple" form searches each `value` expression from left to right
until it finds one that equals `expression`:
```sql theme={null}
CASE expression
WHEN value THEN result
[ WHEN ... ]
[ ELSE result ]
END
```
The `result` for the matching `value` is returned.
If no match is found, the `result` from the `ELSE` clause is
returned if it exists, otherwise null is returned. Example:
```sql theme={null}
SELECT a,
CASE a
WHEN 1 THEN 'one'
WHEN 2 THEN 'two'
ELSE 'many'
END
```
The "searched" form evaluates each boolean `condition` from left
to right until one is true and returns the matching `result`:
```text theme={null}
CASE
WHEN condition THEN result
[ WHEN ... ]
[ ELSE result ]
END
```
If no conditions are true, the `result` from the `ELSE` clause is
returned if it exists, otherwise null is returned. Example:
```sql theme={null}
SELECT a, b,
CASE
WHEN a = 1 THEN 'aaa'
WHEN b = 2 THEN 'bbb'
ELSE 'ccc'
END
```
SQL routines can use [`CASE` statements](/sql/routines/case) that use a slightly
different syntax from the CASE expressions. Specifically note the requirements
for terminating each clause with a semicolon `;` and the usage of `END CASE`.
## IF
The `IF` expression has two forms, one supplying only a
`true_value` and the other supplying both a `true_value` and a
`false_value`:
```
if(condition, true_value)
```
Evaluates and returns `true_value` if `condition` is true,
otherwise null is returned and `true_value` is not evaluated.
```
if(condition, true_value, false_value)
```
Evaluates and returns `true_value` if `condition` is true,
otherwise evaluates and returns `false_value`.
The following `IF` and `CASE` expressions are equivalent:
```sql theme={null}
SELECT
orderkey,
totalprice,
IF(totalprice >= 150000, 'High Value', 'Low Value')
FROM tpch.sf1.orders;
```
```sql theme={null}
SELECT
orderkey,
totalprice,
CASE
WHEN totalprice >= 150000 THEN 'High Value'
ELSE 'Low Value'
END
FROM tpch.sf1.orders;
```
SQL routines can use [`IF` statements](/sql/routines/if) that use a slightly
different syntax from `IF` expressions. Specifically note the requirement
for terminating each clause with a semicolon `;` and the usage of `END IF`.
## COALESCE
```
coalesce(value1, value2[, ...])
```
Returns the first non-null `value` in the argument list.
Like a `CASE` expression, arguments are only evaluated if necessary.
## NULLIF
```
nullif(value1, value2)
```
Returns null if `value1` equals `value2`, otherwise returns `value1`.
## TRY
```
try(expression)
```
Evaluate an expression and handle certain types of errors by returning
`NULL`.
In cases where it is preferable that queries produce `NULL` or default values
instead of failing when corrupt or invalid data is encountered, the `TRY`
function may be useful. To specify default values, the `TRY` function can be
used in conjunction with the `COALESCE` function.
The following errors are handled by `TRY`:
* Division by zero
* Invalid cast or function argument
* Numeric value out of range
### Examples
Source table with some invalid data:
```sql theme={null}
SELECT * FROM shipping;
```
```text theme={null}
origin_state | origin_zip | packages | total_cost
--------------+------------+----------+------------
California | 94131 | 25 | 100
California | P332a | 5 | 72
California | 94025 | 0 | 155
New Jersey | 08544 | 225 | 490
(4 rows)
```
Query failure without `TRY`:
```sql theme={null}
SELECT CAST(origin_zip AS BIGINT) FROM shipping;
```
```text theme={null}
Query failed: Cannot cast 'P332a' to BIGINT
```
`NULL` values with `TRY`:
```sql theme={null}
SELECT TRY(CAST(origin_zip AS BIGINT)) FROM shipping;
```
```text theme={null}
origin_zip
------------
94131
NULL
94025
08544
(4 rows)
```
Query failure without `TRY`:
```sql theme={null}
SELECT total_cost / packages AS per_package FROM shipping;
```
```text theme={null}
Query failed: Division by zero
```
Default values with `TRY` and `COALESCE`:
```sql theme={null}
SELECT COALESCE(TRY(total_cost / packages), 0) AS per_package FROM shipping;
```
```text theme={null}
per_package
-------------
4
14
0
19
(4 rows)
```
# Conversion
Source: https://docs.upsolve.ai/sql/functions/conversion
# Conversion functions
Trino will implicitly convert numeric and character values to the
correct type if such a conversion is possible. Trino will not convert
between character and numeric types. For example, a query that expects
a varchar will not automatically convert a bigint value to an
equivalent varchar.
When necessary, values can be explicitly cast to a particular type.
## Conversion functions
### cast
```
cast(value AS type) -> type
```
Explicitly cast a value as a type. This can be used to cast a
varchar to a numeric value type and vice versa.
### try\_cast
```
try_cast(value AS type) -> type
```
Like [cast](#cast), but returns null if the cast fails.
## Formatting
### format
```
format(format, args...) -> varchar
```
Returns a formatted string using the specified [format string](https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/util/Formatter.html#syntax)
and arguments:
```sql theme={null}
SELECT format('%s%%', 123);
-- '123%'
SELECT format('%.5f', pi());
-- '3.14159'
SELECT format('%03d', 8);
-- '008'
SELECT format('%,.2f', 1234567.89);
-- '1,234,567.89'
SELECT format('%-7s,%7s', 'hello', 'world');
-- 'hello , world'
SELECT format('%2$s %3$s %1$s', 'a', 'b', 'c');
-- 'b c a'
SELECT format('%1$tA, %1$tB %1$te, %1$tY', date '2006-07-04');
-- 'Tuesday, July 4, 2006'
```
### format\_number
```
format_number(number) -> varchar
```
Returns a formatted string using a unit symbol:
```sql theme={null}
SELECT format_number(123456); -- '123K'
SELECT format_number(1000000); -- '1M'
```
## Data size
The `parse_data_size` function supports the following units:
| Unit | Description | Value |
| ---- | ----------- | ---------------- |
| `B` | Bytes | 1 |
| `kB` | Kilobytes | 1024 |
| `MB` | Megabytes | 10242 |
| `GB` | Gigabytes | 10243 |
| `TB` | Terabytes | 10244 |
| `PB` | Petabytes | 10245 |
| `EB` | Exabytes | 10246 |
| `ZB` | Zettabytes | 10247 |
| `YB` | Yottabytes | 10248 |
### parse\_data\_size
```
parse_data_size(string) -> decimal(38)
```
Parses `string` of format `value unit` into a number, where
`value` is the fractional number of `unit` values:
```sql theme={null}
SELECT parse_data_size('1B'); -- 1
SELECT parse_data_size('1kB'); -- 1024
SELECT parse_data_size('1MB'); -- 1048576
SELECT parse_data_size('2.3MB'); -- 2411724
```
## Miscellaneous
### typeof
```
typeof(expr) -> varchar
```
Returns the name of the type of the provided expression:
```sql theme={null}
SELECT typeof(123); -- integer
SELECT typeof('cat'); -- varchar(3)
SELECT typeof(cos(2) + 1.5); -- double
```
# Date and time functions and operators
Source: https://docs.upsolve.ai/sql/functions/datetime
These functions and operators operate on [date and time data types](/sql/reference/types#date-time-data-types).
## Date and time operators
| Operator | Example | Result |
| -------- | --------------------------------------------------- | ------------------------- |
| `+` | `date '2012-08-08' + interval '2' day` | `2012-08-10` |
| `+` | `time '01:00' + interval '3' hour` | `04:00:00.000` |
| `+` | `timestamp '2012-08-08 01:00' + interval '29' hour` | `2012-08-09 06:00:00.000` |
| `+` | `timestamp '2012-10-31 01:00' + interval '1' month` | `2012-11-30 01:00:00.000` |
| `+` | `interval '2' day + interval '3' hour` | `2 03:00:00.000` |
| `+` | `interval '3' year + interval '5' month` | `3-5` |
| `-` | `date '2012-08-08' - interval '2' day` | `2012-08-06` |
| `-` | `time '01:00' - interval '3' hour` | `22:00:00.000` |
| `-` | `timestamp '2012-08-08 01:00' - interval '29' hour` | `2012-08-06 20:00:00.000` |
| `-` | `timestamp '2012-10-31 01:00' - interval '1' month` | `2012-09-30 01:00:00.000` |
| `-` | `interval '2' day - interval '3' hour` | `1 21:00:00.000` |
| `-` | `interval '3' year - interval '5' month` | `2-7` |
## Time zone conversion
The `AT TIME ZONE` operator sets the time zone of a timestamp:
```sql theme={null}
SELECT timestamp '2012-10-31 01:00 UTC';
-- 2012-10-31 01:00:00.000 UTC
SELECT timestamp '2012-10-31 01:00 UTC' AT TIME ZONE 'America/Los_Angeles';
-- 2012-10-30 18:00:00.000 America/Los_Angeles
```
## Date and time functions
### current\_date
```
current_date
```
Returns the current date as of the start of the query.
### current\_time
```
current_time
```
Returns the current time with time zone as of the start of the query.
### current\_timestamp
```
current_timestamp
```
Returns the current timestamp with time zone as of the start of the query,
with `3` digits of subsecond precision,
```
current_timestamp(p)
```
Returns the current [timestamp with time zone](/sql/reference/types#timestamp-with-time-zone-data-type) as of the start of the query, with
`p` digits of subsecond precision:
```sql theme={null}
SELECT current_timestamp(6);
-- 2020-06-24 08:25:31.759993 America/Los_Angeles
```
### current\_timezone
```
current_timezone() -> varchar
```
Returns the current time zone in the format defined by IANA
(e.g., `America/Los_Angeles`) or as fixed offset from UTC (e.g., `+08:35`)
### date
```
date(x) -> date
```
This is an alias for `CAST(x AS date)`.
### last\_day\_of\_month
```
last_day_of_month(x) -> date
```
Returns the last day of the month.
### from\_iso8601\_timestamp
```
from_iso8601_timestamp(string) -> timestamp(3) with time zone
```
Parses the ISO 8601 formatted date `string`, optionally with time and time
zone, into a `timestamp(3) with time zone`. The time defaults to
`00:00:00.000`, and the time zone defaults to the session time zone:
```sql theme={null}
SELECT from_iso8601_timestamp('2020-05-11');
-- 2020-05-11 00:00:00.000 America/Vancouver
SELECT from_iso8601_timestamp('2020-05-11T11:15:05');
-- 2020-05-11 11:15:05.000 America/Vancouver
SELECT from_iso8601_timestamp('2020-05-11T11:15:05.055+01:00');
-- 2020-05-11 11:15:05.055 +01:00
```
### from\_iso8601\_timestamp\_nanos
```
from_iso8601_timestamp_nanos(string) -> timestamp(9) with time zone
```
Parses the ISO 8601 formatted date and time `string`. The time zone
defaults to the session time zone:
```sql theme={null}
SELECT from_iso8601_timestamp_nanos('2020-05-11T11:15:05');
-- 2020-05-11 11:15:05.000000000 America/Vancouver
SELECT from_iso8601_timestamp_nanos('2020-05-11T11:15:05.123456789+01:00');
-- 2020-05-11 11:15:05.123456789 +01:00
```
### from\_iso8601\_date
```
from_iso8601_date(string) -> date
```
Parses the ISO 8601 formatted date `string` into a `date`. The date can
be a calendar date, a week date using ISO week numbering, or year and day
of year combined:
```sql theme={null}
SELECT from_iso8601_date('2020-05-11');
-- 2020-05-11
SELECT from_iso8601_date('2020-W10');
-- 2020-03-02
SELECT from_iso8601_date('2020-123');
-- 2020-05-02
```
### at\_timezone
```
at_timezone(timestamp(p), zone) -> timestamp(p) with time zone
```
Returns the timestamp specified in `timestamp` with the time zone
converted from the session time zone to the time zone specified in `zone`
with precision `p`. In the following example, the session time zone is set
to `America/New_York`, which is three hours ahead of
`America/Los_Angeles`:
```sql theme={null}
SELECT current_timezone()
-- America/New_York
SELECT at_timezone(TIMESTAMP '2022-11-01 09:08:07.321', 'America/Los_Angeles')
-- 2022-11-01 06:08:07.321 America/Los_Angeles
```
### with\_timezone
```
with_timezone(timestamp(p), zone) -> timestamp(p) with time zone
```
Returns the timestamp specified in `timestamp` with the time zone
specified in `zone` with precision `p`:
```sql theme={null}
SELECT current_timezone()
-- America/New_York
SELECT with_timezone(TIMESTAMP '2022-11-01 09:08:07.321', 'America/Los_Angeles')
-- 2022-11-01 09:08:07.321 America/Los_Angeles
```
### from\_unixtime
```
from_unixtime(unixtime) -> timestamp(3) with time zone
```
Returns the UNIX timestamp `unixtime` as a timestamp with time zone. `unixtime` is the
number of seconds since `1970-01-01 00:00:00 UTC`.
```
from_unixtime(unixtime, zone) -> timestamp(3) with time zone
```
Returns the UNIX timestamp `unixtime` as a timestamp with time zone
using `zone` for the time zone. `unixtime` is the number of seconds
since `1970-01-01 00:00:00 UTC`.
```
from_unixtime(unixtime, hours, minutes) -> timestamp(3) with time zone
```
Returns the UNIX timestamp `unixtime` as a timestamp with time zone
using `hours` and `minutes` for the time zone offset. `unixtime` is
the number of seconds since `1970-01-01 00:00:00` in `double` data type.
### from\_unixtime\_nanos
```
from_unixtime_nanos(unixtime) -> timestamp(9) with time zone
```
Returns the UNIX timestamp `unixtime` as a timestamp with time zone. `unixtime` is the
number of nanoseconds since `1970-01-01 00:00:00.000000000 UTC`:
```sql theme={null}
SELECT from_unixtime_nanos(100);
-- 1970-01-01 00:00:00.000000100 UTC
SELECT from_unixtime_nanos(DECIMAL '1234');
-- 1970-01-01 00:00:00.000001234 UTC
SELECT from_unixtime_nanos(DECIMAL '1234.499');
-- 1970-01-01 00:00:00.000001234 UTC
SELECT from_unixtime_nanos(DECIMAL '-1234');
-- 1969-12-31 23:59:59.999998766 UTC
```
### localtime
```
localtime
```
Returns the current time as of the start of the query.
### localtimestamp
```
localtimestamp
```
Returns the current timestamp as of the start of the query, with `3`
digits of subsecond precision.
```
localtimestamp(p)
```
Returns the current [timestamp](/sql/reference/types#timestamp) as of the start
of the query, with `p` digits of subsecond precision:
```sql theme={null}
SELECT localtimestamp(6);
-- 2020-06-10 15:55:23.383628
```
### now
```
now() -> timestamp(3) with time zone
```
This is an alias for `current_timestamp`.
### to\_iso8601
```
to_iso8601(x) -> varchar
```
Formats `x` as an ISO 8601 string. `x` can be date, timestamp, or
timestamp with time zone.
### to\_milliseconds
```
to_milliseconds(interval) -> bigint
```
Returns the day-to-second `interval` as milliseconds.
### to\_unixtime
```
to_unixtime(timestamp) -> double
```
Returns `timestamp` as a UNIX timestamp.
The following SQL-standard functions do not use parenthesis:
* `current_date`
* `current_time`
* `current_timestamp`
* `localtime`
* `localtimestamp`
## Truncation function
The `date_trunc` function supports the following units:
| Unit | Example Truncated Value |
| ------------- | ------------------------- |
| `millisecond` | `2001-08-22 03:04:05.321` |
| `second` | `2001-08-22 03:04:05.000` |
| `minute` | `2001-08-22 03:04:00.000` |
| `hour` | `2001-08-22 03:00:00.000` |
| `day` | `2001-08-22 00:00:00.000` |
| `week` | `2001-08-20 00:00:00.000` |
| `month` | `2001-08-01 00:00:00.000` |
| `quarter` | `2001-07-01 00:00:00.000` |
| `year` | `2001-01-01 00:00:00.000` |
The above examples use the timestamp `2001-08-22 03:04:05.321` as the input.
### date\_trunc
```
date_trunc(unit, x) -> [same as input]
```
Returns `x` truncated to `unit`:
```sql theme={null}
SELECT date_trunc('day' , TIMESTAMP '2022-10-20 05:10:00');
-- 2022-10-20 00:00:00.000
SELECT date_trunc('month' , TIMESTAMP '2022-10-20 05:10:00');
-- 2022-10-01 00:00:00.000
SELECT date_trunc('year', TIMESTAMP '2022-10-20 05:10:00');
-- 2022-01-01 00:00:00.000
```
## Interval functions
The functions in this section support the following interval units:
| Unit | Description |
| ------------- | ------------------ |
| `millisecond` | Milliseconds |
| `second` | Seconds |
| `minute` | Minutes |
| `hour` | Hours |
| `day` | Days |
| `week` | Weeks |
| `month` | Months |
| `quarter` | Quarters of a year |
| `year` | Years |
### date\_add
```
date_add(unit, value, timestamp) -> [same as input]
```
Adds an interval `value` of type `unit` to `timestamp`.
Subtraction can be performed by using a negative value:
```sql theme={null}
SELECT date_add('second', 86, TIMESTAMP '2020-03-01 00:00:00');
-- 2020-03-01 00:01:26.000
SELECT date_add('hour', 9, TIMESTAMP '2020-03-01 00:00:00');
-- 2020-03-01 09:00:00.000
SELECT date_add('day', -1, TIMESTAMP '2020-03-01 00:00:00 UTC');
-- 2020-02-29 00:00:00.000 UTC
```
### date\_diff
```
date_diff(unit, timestamp1, timestamp2) -> bigint
```
Returns `timestamp2 - timestamp1` expressed in terms of `unit`:
```sql theme={null}
SELECT date_diff('second', TIMESTAMP '2020-03-01 00:00:00', TIMESTAMP '2020-03-02 00:00:00');
-- 86400
SELECT date_diff('hour', TIMESTAMP '2020-03-01 00:00:00 UTC', TIMESTAMP '2020-03-02 00:00:00 UTC');
-- 24
SELECT date_diff('day', DATE '2020-03-01', DATE '2020-03-02');
-- 1
SELECT date_diff('second', TIMESTAMP '2020-06-01 12:30:45.000000000', TIMESTAMP '2020-06-02 12:30:45.123456789');
-- 86400
SELECT date_diff('millisecond', TIMESTAMP '2020-06-01 12:30:45.000000000', TIMESTAMP '2020-06-02 12:30:45.123456789');
-- 86400123
```
## Duration function
The `parse_duration` function supports the following units:
| Unit | Description |
| ---- | ------------ |
| `ns` | Nanoseconds |
| `us` | Microseconds |
| `ms` | Milliseconds |
| `s` | Seconds |
| `m` | Minutes |
| `h` | Hours |
| `d` | Days |
### parse\_duration
```
parse_duration(string) -> interval
```
Parses `string` of format `value unit` into an interval, where
`value` is fractional number of `unit` values:
```sql theme={null}
SELECT parse_duration('42.8ms');
-- 0 00:00:00.043
SELECT parse_duration('3.81 d');
-- 3 19:26:24.000
SELECT parse_duration('5m');
-- 0 00:05:00.000
```
### human\_readable\_seconds
```
human_readable_seconds(double) -> varchar
```
Formats the double value of `seconds` into a human readable string containing
`weeks`, `days`, `hours`, `minutes`, and `seconds`:
```sql theme={null}
SELECT human_readable_seconds(96);
-- 1 minute, 36 seconds
SELECT human_readable_seconds(3762);
-- 1 hour, 2 minutes, 42 seconds
SELECT human_readable_seconds(56363463);
-- 93 weeks, 1 day, 8 hours, 31 minutes, 3 seconds
```
## MySQL date functions
The functions in this section use a format string that is compatible with
the MySQL `date_parse` and `str_to_date` functions. The following table,
based on the MySQL manual, describes the format specifiers:
| Specifier | Description |
| --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `%a` | Abbreviated weekday name (`Sun` .. `Sat`) |
| `%b` | Abbreviated month name (`Jan` .. `Dec`) |
| `%c` | Month, numeric (`1` .. `12`), this specifier does not support `0` as a month. |
| `%D` | Day of the month with English suffix (`0th`, `1st`, `2nd`, `3rd`, ...) |
| `%d` | Day of the month, numeric (`01` .. `31`), this specifier does not support `0` as a month or day. |
| `%e` | Day of the month, numeric (`1` .. `31`), this specifier does not support `0` as a day. |
| `%f` | Fraction of second (6 digits for printing: `000000` .. `999000`; 1 - 9 digits for parsing: `0` .. `999999999`), timestamp is truncated to milliseconds. |
| `%H` | Hour (`00` .. `23`) |
| `%h` | Hour (`01` .. `12`) |
| `%I` | Hour (`01` .. `12`) |
| `%i` | Minutes, numeric (`00` .. `59`) |
| `%j` | Day of year (`001` .. `366`) |
| `%k` | Hour (`0` .. `23`) |
| `%l` | Hour (`1` .. `12`) |
| `%M` | Month name (`January` .. `December`) |
| `%m` | Month, numeric (`01` .. `12`), this specifier does not support `0` as a month. |
| `%p` | `AM` or `PM` |
| `%r` | Time of day, 12-hour (equivalent to `%h:%i:%s %p`) |
| `%S` | Seconds (`00` .. `59`) |
| `%s` | Seconds (`00` .. `59`) |
| `%T` | Time of day, 24-hour (equivalent to `%H:%i:%s`) |
| `%U` | Week (`00` .. `53`), where Sunday is the first day of the week |
| `%u` | Week (`00` .. `53`), where Monday is the first day of the week |
| `%V` | Week (`01` .. `53`), where Sunday is the first day of the week; used with `%X` |
| `%v` | Week (`01` .. `53`), where Monday is the first day of the week; used with `%x` |
| `%W` | Weekday name (`Sunday` .. `Saturday`) |
| `%w` | Day of the week (`0` .. `6`), where Sunday is the first day of the week, this specifier is not supported,consider using `day_of_week` (it uses `1-7` instead of `0-6`). |
| `%X` | Year for the week where Sunday is the first day of the week, numeric, four digits; used with `%V` |
| `%x` | Year for the week, where Monday is the first day of the week, numeric, four digits; used with `%v` |
| `%Y` | Year, numeric, four digits |
| `%y` | Year, numeric (two digits), when parsing, two-digit year format assumes range `1970` .. `2069`, so "70" will result in year `1970` but "69" will produce `2069`. |
| `%%` | A literal `%` character |
| `%x` | `x`, for any `x` not listed above |
The following specifiers are not currently supported: `%D %U %u %V %w %X`
### date\_format
```
date_format(timestamp, format) -> varchar
```
Formats `timestamp` as a string using `format`:
```sql theme={null}
SELECT date_format(TIMESTAMP '2022-10-20 05:10:00', '%m-%d-%Y %H');
-- 10-20-2022 05
```
### date\_parse
```
date_parse(string, format) → timestamp(3)
```
Parses `string` into a timestamp using `format`:
```sql theme={null}
SELECT date_parse('2022/10/20/05', '%Y/%m/%d/%H');
-- 2022-10-20 05:00:00.000
```
## Java date functions
The functions in this section use a format string that is compatible with
JodaTime's [DateTimeFormat] pattern format.
### format\_datetime
```
format_datetime(timestamp, format) -> varchar
```
Formats `timestamp` as a string using `format`.
### parse\_datetime
```
parse_datetime(string, format) -> timestamp with time zone
```
Parses `string` into a timestamp with time zone using `format`.
## Extraction function
The `extract` function supports the following fields:
| Field | Description |
| ----------------- | ----------------- |
| `YEAR` | `year` |
| `QUARTER` | `quarter` |
| `MONTH` | `month` |
| `WEEK` | `week` |
| `DAY` | `day` |
| `DAY_OF_MONTH` | `day` |
| `DAY_OF_WEEK` | `day_of_week` |
| `DOW` | `day_of_week` |
| `DAY_OF_YEAR` | `day_of_year` |
| `DOY` | `day_of_year` |
| `YEAR_OF_WEEK` | `year_of_week` |
| `YOW` | `year_of_week` |
| `HOUR` | `hour` |
| `MINUTE` | `minute` |
| `SECOND` | `second` |
| `TIMEZONE_HOUR` | `timezone_hour` |
| `TIMEZONE_MINUTE` | `timezone_minute` |
The types supported by the `extract` function vary depending on the
field to be extracted. Most fields support all date and time types.
### extract
```
extract(field FROM x) -> bigint
```
Returns `field` from `x`:
```sql theme={null}
SELECT extract(YEAR FROM TIMESTAMP '2022-10-20 05:10:00');
-- 2022
```
This SQL-standard function uses special syntax for specifying the arguments.
## Convenience extraction functions
### day
```
day(x) -> bigint
```
Returns the day of the month from `x`.
### day\_of\_month
```
day_of_month(x) -> bigint
```
This is an alias for `day`.
### day\_of\_week
```
day_of_week(x) -> bigint
```
Returns the ISO day of the week from `x`.
The value ranges from `1` (Monday) to `7` (Sunday).
### day\_of\_year
```
day_of_year(x) -> bigint
```
Returns the day of the year from `x`.
The value ranges from `1` to `366`.
### dow
```
dow(x) -> bigint
```
This is an alias for `day_of_week`.
### doy
```
doy(x) -> bigint
```
This is an alias for `day_of_year`.
### hour
```
hour(x) -> bigint
```
Returns the hour of the day from `x`.
The value ranges from `0` to `23`.
### millisecond
```
millisecond(x) -> bigint
```
Returns the millisecond of the second from `x`.
### minute
```
minute(x) -> bigint
```
Returns the minute of the hour from `x`.
### month
```
month(x) -> bigint
```
Returns the month of the year from `x`.
### quarter
```
quarter(x) -> bigint
```
Returns the quarter of the year from `x`.
The value ranges from `1` to `4`.
### second
```
second(x) -> bigint
```
Returns the second of the minute from `x`.
### timezone\_hour
```
timezone_hour(timestamp) -> bigint
```
Returns the hour of the time zone offset from `timestamp`.
### timezone\_minute
```
timezone_minute(timestamp) -> bigint
```
Returns the minute of the time zone offset from `timestamp`.
### week
```
week(x) -> bigint
```
Returns the ISO week of the year from `x`.
The value ranges from `1` to `53`.
### week\_of\_year
```
week_of_year(x) -> bigint
```
This is an alias for [week](#week).
### year
```
year(x) -> bigint
```
Returns the year from `x`.
### year\_of\_week
```
year_of_week(x) -> bigint
```
Returns the year of the \[ISO week] from `x`.
### yow
```
yow(x) -> bigint
```
This is an alias for [year\_of\_week](#year_of_week).
[datetimeformat]: http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html
# Decimal functions and operators
Source: https://docs.upsolve.ai/sql/functions/decimal
## Decimal literals
Use the `DECIMAL 'xxxxxxx.yyyyyyy'` syntax to define a decimal literal.
The precision of a decimal type for a literal will be equal to the number of digits
in the literal (including trailing and leading zeros). The scale will be equal
to the number of digits in the fractional part (including trailing zeros).
| Example literal | Data type |
| --------------------------------- | ----------------- |
| `DECIMAL '0'` | `DECIMAL(1)` |
| `DECIMAL '12345'` | `DECIMAL(5)` |
| `DECIMAL '0000012345.1234500000'` | `DECIMAL(20, 10)` |
## Binary arithmetic decimal operators
Standard mathematical operators are supported. The table below explains
precision and scale calculation rules for result.
Assuming `x` is of type `DECIMAL(xp, xs)` and `y` is of type `DECIMAL(yp, ys)`.
| Operation | Result type precision | Result type scale |
| ------------------- | -------------------------------------------------- | ----------------- |
| `x + y` and `x - y` | `min(38, 1 + max(xs, ys) + max(xp - xs, yp - ys))` | `max(xs, ys)` |
| `x * y` | `min(38, xp + yp)` | `xs + ys` |
| `x / y` | `min(38, xp + ys-xs + max(0, ys-xs))` | `max(xs, ys)` |
| `x % y` | `min(xp - xs, yp - ys) + max(xs, bs)` | `max(xs, ys)` |
If the mathematical result of the operation is not exactly representable with
the precision and scale of the result data type,
then an exception condition is raised: `Value is out of range`.
When operating on decimal types with different scale and precision, the values are
first coerced to a common super type. For types near the largest representable precision (38),
this can result in Value is out of range errors when one of the operands doesn't fit
in the common super type. For example, the common super type of decimal(38, 0) and
decimal(38, 1) is decimal(38, 1), but certain values that fit in decimal(38, 0)
cannot be represented as a decimal(38, 1).
## Comparison operators
All standard [comparison](/sql/functions/comparison#comparison-operators) work for the decimal type.
## Unary decimal operators
The `-` operator performs negation. The type of result is same as type of argument.
# HyperLogLog Functions
Source: https://docs.upsolve.ai/sql/functions/hyperloglog
Trino implements the `approx_distinct` function using the
[HyperLogLog](https://wikipedia.org/wiki/HyperLogLog) data structure.
## Data structures
Trino implements HyperLogLog data sketches as a set of 32-bit buckets which
store a *maximum hash*. They can be stored sparsely (as a map from bucket ID
to bucket), or densely (as a contiguous memory block). The HyperLogLog data
structure starts as the sparse representation, switching to dense when it is
more efficient. The P4HyperLogLog structure is initialized densely and
remains dense for its lifetime.
[HyperLogLog](/sql/reference/types#hyperloglog) implicitly casts to [P4HyperLogLog](/sql/reference/types#p4hyperloglog),
while one can explicitly cast `HyperLogLog` to `P4HyperLogLog`:
```sql theme={null}
cast(hll AS P4HyperLogLog)
```
## Serialization
Data sketches can be serialized to and deserialized from `varbinary`. This
allows them to be stored for later use. Combined with the ability to merge
multiple sketches, this allows one to calculate `approx_distinct` of the
elements of a partition of a query, then for the entirety of a query with very
little cost.
For example, calculating the `HyperLogLog` for daily unique users will allow
weekly or monthly unique users to be calculated incrementally by combining the
dailies. This is similar to computing weekly revenue by summing daily revenue.
Uses of `approx_distinct` with `GROUPING SETS` can be converted to use
`HyperLogLog`. Examples:
```sql theme={null}
CREATE TABLE visit_summaries (
visit_date date,
hll varbinary
);
INSERT INTO visit_summaries
SELECT visit_date, cast(approx_set(user_id) AS varbinary)
FROM user_visits
GROUP BY visit_date;
SELECT cardinality(merge(cast(hll AS HyperLogLog))) AS weekly_unique_users
FROM visit_summaries
WHERE visit_date >= current_date - interval '7' day;
```
## Functions
### approx\_set
```text theme={null}
approx_set(x) -> HyperLogLog
```
Returns the `HyperLogLog` sketch of the input data set of `x`. This
data sketch underlies `approx_distinct` and can be stored and
used later by calling `cardinality()`.
### cardinality
```text theme={null}
cardinality(hll) -> bigint
```
This will perform `approx_distinct` on the data summarized by the
`hll` HyperLogLog data sketch.
### empty\_approx\_set
```text theme={null}
empty_approx_set() -> HyperLogLog
```
Returns an empty `HyperLogLog`.
### merge
```text theme={null}
merge(HyperLogLog) -> HyperLogLog
```
Returns the `HyperLogLog` of the aggregate union of the individual `hll`
HyperLogLog structures.
# JSON functions and operators
Source: https://docs.upsolve.ai/sql/functions/json
The SQL standard describes functions and operators to process JSON data. They
allow you to access JSON data according to its structure, generate JSON data,
and store it persistently in SQL tables.
Importantly, the SQL standard imposes that there is no dedicated data type to
represent JSON data in SQL. Instead, JSON data is represented as character or
binary strings. Although Trino supports `JSON` type, it is not used or
produced by the following functions.
Trino supports three functions for querying JSON data:
[json\_exists](#json-exists), [json\_query](#json-query), and [json\_value](#json-value). Each of them
is based on the same mechanism of exploring and processing JSON input using
JSON path.
Trino also supports two functions for generating JSON data:
[json-array](#json-array), and [json\_object](#json-object).
## JSON path language
The JSON path language is a special language, used exclusively by certain SQL
operators to specify the query to perform on the JSON input. Although JSON path
expressions are embedded in SQL queries, their syntax significantly differs
from SQL. The semantics of predicates, operators, etc. in JSON path expressions
generally follow the semantics of SQL. The JSON path language is case-sensitive
for keywords and identifiers.
### JSON path syntax and semantics
JSON path expressions are recursive structures. Although the name "path"
suggests a linear sequence of operations going step by step deeper into the JSON
structure, a JSON path expression is in fact a tree. It can access the input
JSON item multiple times, in multiple ways, and combine the results. Moreover,
the result of a JSON path expression is not a single item, but an ordered
sequence of items. Each of the sub-expressions takes one or more input
sequences, and returns a sequence as the result.
In the lax mode, most path operations first unnest all JSON arrays in the
input sequence. Any divergence from this rule is mentioned in the following
listing. Path modes are explained in [JSON path modes](#json-path-modes).
The JSON path language features are divided into: literals, variables,
arithmetic binary expressions, arithmetic unary expressions, and a group of
operators collectively known as accessors.
#### literals
* numeric literals
They include exact and approximate numbers, and are interpreted as if they
were SQL values.
```
-1, 1.2e3, NaN
```
* string literals
They are enclosed in double quotes.
```
"Some text"
```
* boolean literals
```
true, false
```
* null literal
It has the semantics of the JSON null, not of SQL null. See [comparison rules](#json-comparison-rules).
```
null
```
#### variables
* context variable
It refers to the currently processed input of the JSON
function.
```
$
```
* named variable
It refers to a named parameter by its name.
```
$param
```
* current item variable
It is used inside the filter expression to refer to the currently processed
item from the input sequence.
```
@
```
* last subscript variable
It refers to the last index of the innermost enclosing array. Array indexes
in JSON path expressions are zero-based.
```
last
```
#### arithmetic binary expressions
The JSON path language supports five arithmetic binary operators:
```
+
-
*
/
%
```
Both operands, `` and ``, are evaluated to sequences of
items. For arithmetic binary operators, each input sequence must contain a
single numeric item. The arithmetic operation is performed according to SQL
semantics, and it returns a sequence containing a single element with the
result.
The operators follow the same precedence rules as in SQL arithmetic operations,
and parentheses can be used for grouping.
#### arithmetic unary expressions
```
+
-
```
The operand `` is evaluated to a sequence of items. Every item must be
a numeric value. The unary plus or minus is applied to every item in the
sequence, following SQL semantics, and the results form the returned sequence.
#### member accessor
The member accessor returns the value of the member with the specified key for
each JSON object in the input sequence.
```
.key
."key"
```
The condition when a JSON object does not have such a member is called a
structural error. In the lax mode, it is suppressed, and the faulty object is
excluded from the result.
Let `` return a sequence of three JSON objects:
```
{"customer" : 100, "region" : "AFRICA"},
{"region" : "ASIA"},
{"customer" : 300, "region" : "AFRICA", "comment" : null}
```
the expression `.customer` succeeds in the first and the third object,
but the second object lacks the required member. In strict mode, path
evaluation fails. In lax mode, the second object is silently skipped, and the
resulting sequence is `100, 300`.
All items in the input sequence must be JSON objects.
Trino does not support JSON objects with duplicate keys.
#### wildcard member accessor
Returns values from all key-value pairs for each JSON object in the input
sequence. All the partial results are concatenated into the returned sequence.
```
.*
```
Let `` return a sequence of three JSON objects:
```
{"customer" : 100, "region" : "AFRICA"},
{"region" : "ASIA"},
{"customer" : 300, "region" : "AFRICA", "comment" : null}
```
The results is:
```
100, "AFRICA", "ASIA", 300, "AFRICA", null
```
All items in the input sequence must be JSON objects.
The order of values returned from a single JSON object is arbitrary. The
sub-sequences from all JSON objects are concatenated in the same order in which
the JSON objects appear in the input sequence.
#### descendant member accessor
Returns the values associated with the specified key in all JSON objects on all
levels of nesting in the input sequence.
```
..key
.."key"
```
The order of returned values is that of preorder depth first search. First, the
enclosing object is visited, and then all child nodes are visited.
This method does not perform array unwrapping in the lax mode. The results
are the same in the lax and strict modes. The method traverses into JSON
arrays and JSON objects. Non-structural JSON items are skipped.
Let `` be a sequence containing a JSON object:
```
{
"id" : 1,
"notes" : [{"type" : 1, "comment" : "foo"}, {"type" : 2, "comment" : null}],
"comment" : ["bar", "baz"]
}
```
```
..comment --> ["bar", "baz"], "foo", null
```
#### array accessor
Returns the elements at the specified indexes for each JSON array in the input
sequence. Indexes are zero-based.
```
[ ]
```
The `` list contains one or more subscripts. Each subscript
specifies a single index or a range (ends inclusive):
```
[, to , ,...]
```
In lax mode, any non-array items resulting from the evaluation of the input
sequence are wrapped into single-element arrays. Note that this is an exception
to the rule of automatic array wrapping.
Each array in the input sequence is processed in the following way:
* The variable `last` is set to the last index of the array.
* All subscript indexes are computed in order of declaration. For a
singleton subscript ``, the result must be a singleton numeric item.
For a range subscript ` to `, two numeric items are expected.
* The specified array elements are added in order to the output sequence.
Let `` return a sequence of three JSON arrays:
```
[0, 1, 2], ["a", "b", "c", "d"], [null, null]
```
The following expression returns a sequence containing the last element from
every array:
```
[last] --> 2, "d", null
```
The following expression returns the third and fourth element from every array:
```
[2 to 3] --> 2, "c", "d"
```
Note that the first array does not have the fourth element, and the last array
does not have the third or fourth element. Accessing non-existent elements is a
structural error. In strict mode, it causes the path expression to fail. In lax
mode, such errors are suppressed, and only the existing elements are returned.
Another example of a structural error is an improper range specification such
as `5 to 3`.
Note that the subscripts may overlap, and they do not need to follow the
element order. The order in the returned sequence follows the subscripts:
```
[1, 0, 0] --> 1, 0, 0, "b", "a", "a", null, null, null
```
#### wildcard array accessor
Returns all elements of each JSON array in the input sequence.
```
[*]
```
In lax mode, any non-array items resulting from the evaluation of the input
sequence are wrapped into single-element arrays. Note that this is an exception
to the rule of automatic array wrapping.
The output order follows the order of the original JSON arrays. Also, the order
of elements within the arrays is preserved.
Let `` return a sequence of three JSON arrays:
```
[0, 1, 2], ["a", "b", "c", "d"], [null, null]
[*] --> 0, 1, 2, "a", "b", "c", "d", null, null
```
#### filter
Retrieves the items from the input sequence which satisfy the predicate.
```
?( )
```
JSON path predicates are syntactically similar to boolean expressions in SQL.
However, the semantics are different in many aspects:
* They operate on sequences of items.
* They have their own error handling (they never fail).
* They behave different depending on the lax or strict mode.
The predicate evaluates to `true`, `false`, or `unknown`. Note that some
predicate expressions involve nested JSON path expression. When evaluating the
nested path, the variable `@` refers to the currently examined item from the
input sequence.
The following predicate expressions are supported:
* Conjunction
```
&&
```
* Disjunction
```
||
```
* Negation
```
!
```
* `exists` predicate
```
exists( )
```
Returns `true` if the nested path evaluates to a non-empty sequence, and
`false` when the nested path evaluates to an empty sequence. If the path
evaluation throws an error, returns `unknown`.
* `starts with` predicate
```
starts with "Some text"
starts with $variable
```
The nested `` must evaluate to a sequence of textual items, and the
other operand must evaluate to a single textual item. If evaluating of either
operand throws an error, the result is `unknown`. All items from the sequence
are checked for starting with the right operand. The result is `true` if a
match is found, otherwise `false`. However, if any of the comparisons throws
an error, the result in the strict mode is `unknown`. The result in the lax
mode depends on whether the match or the error was found first.
* `is unknown` predicate
```
( ) is unknown
```
Returns `true` if the nested predicate evaluates to `unknown`, and
`false` otherwise.
* Comparisons
```
==
<>
!=
<
>
<=
>=
```
Both operands of a comparison evaluate to sequences of items. If either
evaluation throws an error, the result is `unknown`. Items from the left and
right sequence are then compared pairwise. Similarly to the `starts with`
predicate, the result is `true` if any of the comparisons returns `true`,
otherwise `false`. However, if any of the comparisons throws an error, for
example because the compared types are not compatible, the result in the strict
mode is `unknown`. The result in the lax mode depends on whether the `true`
comparison or the error was found first.
##### Comparison rules
Null values in the context of comparison behave different than SQL null:
* null == null --> `true`
* null != null, null \< null, ... --> `false`
* null compared to a scalar value --> `false`
* null compared to a JSON array or a JSON object --> `false`
When comparing two scalar values, `true` or `false` is returned if the
comparison is successfully performed. The semantics of the comparison is the
same as in SQL. In case of an error, e.g. comparing text and number,
`unknown` is returned.
Comparing a scalar value with a JSON array or a JSON object, and comparing JSON
arrays/objects is an error, so `unknown` is returned.
##### Examples of filter
Let `` return a sequence of three JSON objects:
```
{"customer" : 100, "region" : "AFRICA"},
{"region" : "ASIA"},
{"customer" : 300, "region" : "AFRICA", "comment" : null}
```
```
?(@.region != "ASIA") --> {"customer" : 100, "region" : "AFRICA"},
{"customer" : 300, "region" : "AFRICA", "comment" : null}
?(!exists(@.customer)) --> {"region" : "ASIA"}
```
The following accessors are collectively referred to as **item methods**.
#### double()
Converts numeric or text values into double values.
```
.double()
```
Let `` return a sequence `-1, 23e4, "5.6"`:
```
.double() --> -1e0, 23e4, 5.6e0
```
#### ceiling(), floor(), and abs()
Gets the ceiling, the floor or the absolute value for every numeric item in the
sequence. The semantics of the operations is the same as in SQL.
Let `` return a sequence `-1.5, -1, 1.3`:
```
.ceiling() --> -1.0, -1, 2.0
.floor() --> -2.0, -1, 1.0
.abs() --> 1.5, 1, 1.3
```
#### keyvalue()
Returns a collection of JSON objects including one object per every member of
the original object for every JSON object in the sequence.
```
.keyvalue()
```
The returned objects have three members:
* "name", which is the original key,
* "value", which is the original bound value,
* "id", which is the unique number, specific to an input object.
Let `` be a sequence of three JSON objects:
```
{"customer" : 100, "region" : "AFRICA"},
{"region" : "ASIA"},
{"customer" : 300, "region" : "AFRICA", "comment" : null}
```
```
.keyvalue() --> {"name" : "customer", "value" : 100, "id" : 0},
{"name" : "region", "value" : "AFRICA", "id" : 0},
{"name" : "region", "value" : "ASIA", "id" : 1},
{"name" : "customer", "value" : 300, "id" : 2},
{"name" : "region", "value" : "AFRICA", "id" : 2},
{"name" : "comment", "value" : null, "id" : 2}
```
It is required that all items in the input sequence are JSON objects.
The order of the returned values follows the order of the original JSON
objects. However, within objects, the order of returned entries is arbitrary.
#### type()
Returns a textual value containing the type name for every item in the
sequence.
```
.type()
```
This method does not perform array unwrapping in the lax mode.
The returned values are:
* `"null"` for JSON null,
* `"number"` for a numeric item,
* `"string"` for a textual item,
* `"boolean"` for a boolean item,
* `"date"` for an item of type date,
* `"time without time zone"` for an item of type time,
* `"time with time zone"` for an item of type time with time zone,
* `"timestamp without time zone"` for an item of type timestamp,
* `"timestamp with time zone"` for an item of type timestamp with time zone,
* `"array"` for JSON array,
* `"object"` for JSON object,
#### size()
Returns a numeric value containing the size for every JSON array in the
sequence.
```
.size()
```
This method does not perform array unwrapping in the lax mode. Instead, all
non-array items are wrapped in singleton JSON arrays, so their size is `1`.
It is required that all items in the input sequence are JSON arrays.
Let `` return a sequence of three JSON arrays:
```
[0, 1, 2], ["a", "b", "c", "d"], [null, null]
.size() --> 3, 4, 2
```
### Limitations
The SQL standard describes the `datetime()` JSON path item method and the
`like_regex()` JSON path predicate. Trino does not support them.
### JSON path modes
The JSON path expression can be evaluated in two modes: strict and lax. In the
strict mode, it is required that the input JSON data strictly fits the schema
required by the path expression. In the lax mode, the input JSON data can
diverge from the expected schema.
The following table shows the differences between the two modes.
| Condition | strict mode | lax mode |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | -------------------------------------------------------------------------------------------------------------- |
| Performing an operation which requires a non-array on an array, e.g.:
`$.key` requires a JSON object
`$.floor()` requires a numeric value | ERROR | The array is automatically unnested, and the operation is performed on each array element. |
| Performing an operation which requires an array on an non-array, e.g.:
`$[0]`, `$[*]`, `$.size()` | ERROR | The non-array item is automatically wrapped in a singleton array, and the operation is performed on the array. |
| A structural error: accessing a non-existent element of an array or a non-existent member of a JSON object, e.g.:
`$[-1]` (array index out of bounds)
`$.key`, where the input JSON object does not have a member `key` | ERROR | The error is suppressed, and the operation results in an empty sequence. |
#### Examples of the lax mode behavior
Let `` return a sequence of three items, a JSON array, a JSON object,
and a scalar numeric value:
```
[1, "a", null], {"key1" : 1.0, "key2" : true}, -2e3
```
The following example shows the wildcard array accessor in the lax mode. The
JSON array returns all its elements, while the JSON object and the number are
wrapped in singleton arrays and then unnested, so effectively they appear
unchanged in the output sequence:
```
[*] --> 1, "a", null, {"key1" : 1.0, "key2" : true}, -2e3
```
When calling the `size()` method, the JSON object and the number are also
wrapped in singleton arrays:
```
.size() --> 3, 1, 1
```
In some cases, the lax mode cannot prevent failure. In the following example,
even though the JSON array is unwrapped prior to calling the `floor()`
method, the item `"a"` causes type mismatch.
```
.floor() --> ERROR
```
## json\_exists
The `json_exists` function determines whether a JSON value satisfies a JSON
path specification.
```sql theme={null}
JSON_EXISTS(
json_input [ FORMAT JSON [ ENCODING { UTF8 | UTF16 | UTF32 } ] ],
json_path
[ PASSING json_argument [, ...] ]
[ { TRUE | FALSE | UNKNOWN | ERROR } ON ERROR ]
)
```
The `json_path` is evaluated using the `json_input` as the context variable
(`$`), and the passed arguments as the named variables (`$variable_name`).
The returned value is `true` if the path returns a non-empty sequence, and
`false` if the path returns an empty sequence. If an error occurs, the
returned value depends on the `ON ERROR` clause. The default value returned
`ON ERROR` is `FALSE`. The `ON ERROR` clause is applied for the following
kinds of errors:
* Input conversion errors, such as malformed JSON
* JSON path evaluation errors, e.g. division by zero
`json_input` is a character string or a binary string. It should contain
a single JSON item. For a binary string, you can specify encoding.
`json_path` is a string literal, containing the path mode specification, and
the path expression, following the syntax rules described in
[JSON path syntax and semantics](#json-path-syntax-and-semantics).
```
'strict ($.price + $.tax)?(@ > 99.9)'
'lax $[0 to 1].floor()?(@ > 10)'
```
In the `PASSING` clause you can pass arbitrary expressions to be used by the
path expression.
```sql theme={null}
PASSING orders.totalprice AS O_PRICE,
orders.tax % 10 AS O_TAX
```
The passed parameters can be referenced in the path expression by named
variables, prefixed with `$`.
```
'lax $?(@.price > $O_PRICE || @.tax > $O_TAX)'
```
Additionally to SQL values, you can pass JSON values, specifying the format and
optional encoding:
```sql theme={null}
PASSING orders.json_desc FORMAT JSON AS o_desc,
orders.binary_record FORMAT JSON ENCODING UTF16 AS o_rec
```
Note that the JSON path language is case-sensitive, while the unquoted SQL
identifiers are upper-cased. Therefore, it is recommended to use quoted
identifiers in the `PASSING` clause:
```
'lax $.$KeyName' PASSING nation.name AS KeyName --> ERROR; no passed value found
'lax $.$KeyName' PASSING nation.name AS "KeyName" --> correct
```
### Examples
Let `customers` be a table containing two columns: `id:bigint`,
`description:varchar`.
The following query checks which customers have children above the age of 10:
```
SELECT
id,
json_exists(
description,
'lax $.children[*]?(@ > 10)'
) AS children_above_ten
FROM customers
```
| id | children\_above\_ten |
| --- | -------------------- |
| 101 | true |
| 102 | true |
| 103 | false |
In the following query, the path mode is strict. We check the third child for
each customer. This should cause a structural error for the customers who do
not have three or more children. This error is handled according to the `ON
ERROR` clause.
```
SELECT
id,
json_exists(
description,
'strict $.children[2]?(@ > 10)'
UNKNOWN ON ERROR
) AS child_3_above_ten
FROM customers
```
| id | child\_3\_above\_ten |
| --- | -------------------- |
| 101 | true |
| 102 | NULL |
| 103 | NULL |
## json\_query
The `json_query` function extracts a JSON value from a JSON value.
```sql theme={null}
JSON_QUERY(
json_input [ FORMAT JSON [ ENCODING { UTF8 | UTF16 | UTF32 } ] ],
json_path
[ PASSING json_argument [, ...] ]
[ RETURNING type [ FORMAT JSON [ ENCODING { UTF8 | UTF16 | UTF32 } ] ] ]
[ WITHOUT [ ARRAY ] WRAPPER |
WITH [ { CONDITIONAL | UNCONDITIONAL } ] [ ARRAY ] WRAPPER ]
[ { KEEP | OMIT } QUOTES [ ON SCALAR STRING ] ]
[ { ERROR | NULL | EMPTY ARRAY | EMPTY OBJECT } ON EMPTY ]
[ { ERROR | NULL | EMPTY ARRAY | EMPTY OBJECT } ON ERROR ]
)
```
The constant string `json_path` is evaluated using the `json_input` as the
context variable (`$`), and the passed arguments as the named variables
(`$variable_name`).
The returned value is a JSON item returned by the path. By default, it is
represented as a character string (`varchar`). In the `RETURNING` clause,
you can specify other character string type or `varbinary`. With
`varbinary`, you can also specify the desired encoding.
`json_input` is a character string or a binary string. It should contain
a single JSON item. For a binary string, you can specify encoding.
`json_path` is a string literal, containing the path mode specification, and
the path expression, following the syntax rules described in
[JSON path syntax and semantics](#json-path-syntax-and-semantics).
```
'strict $.keyvalue()?(@.name == $cust_id)'
'lax $[5 to last]'
```
In the `PASSING` clause you can pass arbitrary expressions to be used by the
path expression.
```sql theme={null}
PASSING orders.custkey AS CUST_ID
```
The passed parameters can be referenced in the path expression by named
variables, prefixed with `$`.
```
'strict $.keyvalue()?(@.value == $CUST_ID)'
```
Additionally to SQL values, you can pass JSON values, specifying the format and
optional encoding:
```sql theme={null}
PASSING orders.json_desc FORMAT JSON AS o_desc,
orders.binary_record FORMAT JSON ENCODING UTF16 AS o_rec
```
Note that the JSON path language is case-sensitive, while the unquoted SQL
identifiers are upper-cased. Therefore, it is recommended to use quoted
identifiers in the `PASSING` clause:
```
'lax $.$KeyName' PASSING nation.name AS KeyName --> ERROR; no passed value found
'lax $.$KeyName' PASSING nation.name AS "KeyName" --> correct
```
The `ARRAY WRAPPER` clause lets you modify the output by wrapping the results
in a JSON array. `WITHOUT ARRAY WRAPPER` is the default option. `WITH
CONDITIONAL ARRAY WRAPPER` wraps every result which is not a singleton JSON
array or JSON object. `WITH UNCONDITIONAL ARRAY WRAPPER` wraps every result.
The `QUOTES` clause lets you modify the result for a scalar string by
removing the double quotes being part of the JSON string representation.
### Examples
Let `customers` be a table containing two columns: `id:bigint`,
`description:varchar`.
```
| id | description |
| --- | ----------------------------------------------------- |
| 101 | '{"comment" : "nice", "children" : \[10, 13, 16\]}' |
| 102 | '{"comment" : "problematic", "children" : \[8, 11\]}' |
| 103 | '{"comment" : "knows best", "children" : \[2\]}' |
```
The following query gets the `children` array for each customer:
```sql theme={null}
SELECT
id,
json_query(
description,
'lax $.children'
) AS children
FROM customers
```
| id | children |
| --- | ------------- |
| 101 | '\[10,13,16]' |
| 102 | '\[8,11]' |
| 103 | '\[2]' |
The following query gets the collection of children for each customer.
Note that the `json_query` function can only output a single JSON item. If
you don't use array wrapper, you get an error for every customer with multiple
children. The error is handled according to the `ON ERROR` clause.
```sql theme={null}
SELECT
id,
json_query(
description,
'lax $.children[*]'
WITHOUT ARRAY WRAPPER
NULL ON ERROR
) AS children
FROM customers
```
| id | children |
| --- | -------- |
| 101 | NULL |
| 102 | NULL |
| 103 | '2' |
The following query gets the last child for each customer, wrapped in a JSON
array:
```sql theme={null}
SELECT
id,
json_query(
description,
'lax $.children[last]'
WITH ARRAY WRAPPER
) AS last_child
FROM customers
```
| id | last\_child |
| --- | ----------- |
| 101 | '\[16]' |
| 102 | '\[11]' |
| 103 | '\[2]' |
The following query gets all children above the age of 12 for each customer,
wrapped in a JSON array. The second and the third customer don't have children
of this age. Such case is handled according to the `ON EMPTY` clause. The
default value returned `ON EMPTY` is `NULL`. In the following example,
`EMPTY ARRAY ON EMPTY` is specified.
```sql theme={null}
SELECT
id,
json_query(
description,
'strict $.children[*]?(@ > 12)'
WITH ARRAY WRAPPER
EMPTY ARRAY ON EMPTY
) AS children
FROM customers
```
| id | children |
| --- | ---------- |
| 101 | '\[13,16]' |
| 102 | '\[]' |
| 103 | '\[]' |
The following query shows the result of the `QUOTES` clause. Note that `KEEP
QUOTES` is the default.
```sql theme={null}
SELECT
id,
json_query(description, 'strict $.comment' KEEP QUOTES) AS quoted_comment,
json_query(description, 'strict $.comment' OMIT QUOTES) AS unquoted_comment
FROM customers
```
| id | quoted\_comment | unquoted\_comment |
| --- | --------------- | ----------------- |
| 101 | '"nice"' | 'nice' |
| 102 | '"problematic"' | 'problematic' |
| 103 | '"knows best"' | 'knows best' |
If an error occurs, the returned value depends on the `ON ERROR` clause. The
default value returned `ON ERROR` is `NULL`. One example of error is
multiple items returned by the path. Other errors caught and handled according
to the `ON ERROR` clause are:
* Input conversion errors, such as malformed JSON
* JSON path evaluation errors, e.g. division by zero
* Output conversion errors
## json\_value
The `json_value` function extracts a scalar SQL value from a JSON value.
```sql theme={null}
JSON_VALUE(
json_input [ FORMAT JSON [ ENCODING { UTF8 | UTF16 | UTF32 } ] ],
json_path
[ PASSING json_argument [, ...] ]
[ RETURNING type ]
[ { ERROR | NULL | DEFAULT expression } ON EMPTY ]
[ { ERROR | NULL | DEFAULT expression } ON ERROR ]
)
```
The `json_path` is evaluated using the `json_input` as the context variable
(`$`), and the passed arguments as the named variables (`$variable_name`).
The returned value is the SQL scalar returned by the path. By default, it is
converted to string (`varchar`). In the `RETURNING` clause, you can specify
other desired type: a character string type, numeric, boolean or datetime type.
`json_input` is a character string or a binary string. It should contain
a single JSON item. For a binary string, you can specify encoding.
`json_path` is a string literal, containing the path mode specification, and
the path expression, following the syntax rules described in
[JSON path syntax and semantics](#json-path-syntax-and-semantics).
```
'strict $.price + $tax'
'lax $[last].abs().floor()'
```
In the `PASSING` clause you can pass arbitrary expressions to be used by the
path expression.
```sql theme={null}
PASSING orders.tax AS O_TAX
```
The passed parameters can be referenced in the path expression by named
variables, prefixed with `$`.
```
'strict $[last].price + $O_TAX'
```
Additionally to SQL values, you can pass JSON values, specifying the format and
optional encoding:
```
PASSING orders.json_desc FORMAT JSON AS o_desc,
orders.binary_record FORMAT JSON ENCODING UTF16 AS o_rec
```
Note that the JSON path language is case-sensitive, while the unquoted SQL
identifiers are upper-cased. Therefore, it is recommended to use quoted
identifiers in the `PASSING` clause:
```
'lax $.$KeyName' PASSING nation.name AS KeyName --> ERROR; no passed value found
'lax $.$KeyName' PASSING nation.name AS "KeyName" --> correct
```
If the path returns an empty sequence, the `ON EMPTY` clause is applied. The
default value returned `ON EMPTY` is `NULL`. You can also specify the
default value:
```
DEFAULT -1 ON EMPTY
```
If an error occurs, the returned value depends on the `ON ERROR` clause. The
default value returned `ON ERROR` is `NULL`. One example of error is
multiple items returned by the path. Other errors caught and handled according
to the `ON ERROR` clause are:
* Input conversion errors, such as malformed JSON
* JSON path evaluation errors, e.g. division by zero
* Returned scalar not convertible to the desired type
### Examples
Let `customers` be a table containing two columns: `id:bigint`,
`description:varchar`.
```
| id | description |
| --- | ----------------------------------------------------- |
| 101 | '{"comment" : "nice", "children" : \[10, 13, 16\]}' |
| 102 | '{"comment" : "problematic", "children" : \[8, 11\]}' |
| 103 | '{"comment" : "knows best", "children" : \[2\]}' |
```
The following query gets the `comment` for each customer as `char(12)`:
```sql theme={null}
SELECT id, json_value(
description,
'lax $.comment'
RETURNING char(12)
) AS comment
FROM customers
```
| id | comment |
| --- | -------------- |
| 101 | 'nice ' |
| 102 | 'problematic ' |
| 103 | 'knows best ' |
The following query gets the first child's age for each customer as
`tinyint`:
```sql theme={null}
SELECT id, json_value(
description,
'lax $.children[0]'
RETURNING tinyint
) AS child
FROM customers
```
| id | child |
| --- | ----- |
| 101 | 10 |
| 102 | 8 |
| 103 | 2 |
The following query gets the third child's age for each customer. In the strict
mode, this should cause a structural error for the customers who do not have
the third child. This error is handled according to the `ON ERROR` clause.
```sql theme={null}
SELECT id, json_value(
description,
'strict $.children[2]'
DEFAULT 'err' ON ERROR
) AS child
FROM customers
```
| id | child |
| --- | ----- |
| 101 | '16' |
| 102 | 'err' |
| 103 | 'err' |
After changing the mode to lax, the structural error is suppressed, and the
customers without a third child produce empty sequence. This case is handled
according to the `ON EMPTY` clause.
```sql theme={null}
SELECT id, json_value(
description,
'lax $.children[2]'
DEFAULT 'missing' ON EMPTY
) AS child
FROM customers
```
| id | child |
| --- | --------- |
| 101 | '16' |
| 102 | 'missing' |
| 103 | 'missing' |
## json\_array
The `json_array` function creates a JSON array containing given elements.
```sql theme={null}
JSON_ARRAY(
[ array_element [, ...]
[ { NULL ON NULL | ABSENT ON NULL } ] ],
[ RETURNING type [ FORMAT JSON [ ENCODING { UTF8 | UTF16 | UTF32 } ] ] ]
)
```
### Argument types
The array elements can be arbitrary expressions. Each passed value is converted
into a JSON item according to its type, and optional `FORMAT` and
`ENCODING` specification.
You can pass SQL values of types boolean, numeric, and character string. They
are converted to corresponding JSON literals:
```sql theme={null}
SELECT json_array(true, 12e-1, 'text')
--> '[true,1.2,"text"]'
```
Additionally to SQL values, you can pass JSON values. They are character or
binary strings with a specified format and optional encoding:
```sql theme={null}
SELECT json_array(
'[ "text" ] ' FORMAT JSON,
X'5B0035005D00' FORMAT JSON ENCODING UTF16
)
--> '[["text"],[5]]'
```
You can also nest other JSON-returning functions. In that case, the `FORMAT`
option is implicit:
```sql theme={null}
SELECT json_array(
json_query('{"key" : [ "value" ]}', 'lax $.key')
)
--> '[["value"]]'
```
Other passed values are cast to varchar, and they become JSON text literals:
```sql theme={null}
SELECT json_array(
DATE '2001-01-31',
UUID '12151fd2-7586-11e9-8f9e-2a86e4085a59'
)
--> '["2001-01-31","12151fd2-7586-11e9-8f9e-2a86e4085a59"]'
```
You can omit the arguments altogether to get an empty array:
```sql theme={null}
SELECT json_array() --> '[]'
```
### Null handling
If a value passed for an array element is `null`, it is treated according to
the specified null treatment option. If `ABSENT ON NULL` is specified, the
null element is omitted in the result. If `NULL ON NULL` is specified, JSON
`null` is added to the result. `ABSENT ON NULL` is the default
configuration:
```sql theme={null}
SELECT json_array(true, null, 1)
--> '[true,1]'
SELECT json_array(true, null, 1 ABSENT ON NULL)
--> '[true,1]'
SELECT json_array(true, null, 1 NULL ON NULL)
--> '[true,null,1]'
```
### Returned type
The SQL standard imposes that there is no dedicated data type to represent JSON
data in SQL. Instead, JSON data is represented as character or binary strings.
By default, the `json_array` function returns varchar containing the textual
representation of the JSON array. With the `RETURNING` clause, you can
specify other character string type:
```sql theme={null}
SELECT json_array(true, 1 RETURNING VARCHAR(100))
--> '[true,1]'
```
You can also specify to use varbinary and the required encoding as return type.
The default encoding is UTF8:
```sql theme={null}
SELECT json_array(true, 1 RETURNING VARBINARY)
--> X'5b 74 72 75 65 2c 31 5d'
SELECT json_array(true, 1 RETURNING VARBINARY FORMAT JSON ENCODING UTF8)
--> X'5b 74 72 75 65 2c 31 5d'
SELECT json_array(true, 1 RETURNING VARBINARY FORMAT JSON ENCODING UTF16)
--> X'5b 00 74 00 72 00 75 00 65 00 2c 00 31 00 5d 00'
SELECT json_array(true, 1 RETURNING VARBINARY FORMAT JSON ENCODING UTF32)
--> X'5b 00 00 00 74 00 00 00 72 00 00 00 75 00 00 00 65 00 00 00 2c 00 00 00 31 00 00 00 5d 00 00 00'
```
## json\_object
The `json_object` function creates a JSON object containing given key-value pairs.
```sql theme={null}
JSON_OBJECT(
[ key_value [, ...]
[ { NULL ON NULL | ABSENT ON NULL } ] ],
[ { WITH UNIQUE [ KEYS ] | WITHOUT UNIQUE [ KEYS ] } ]
[ RETURNING type [ FORMAT JSON [ ENCODING { UTF8 | UTF16 | UTF32 } ] ] ]
)
```
### Argument passing conventions
There are two conventions for passing keys and values:
```sql theme={null}
SELECT json_object('key1' : 1, 'key2' : true)
--> '{"key1":1,"key2":true}'
SELECT json_object(KEY 'key1' VALUE 1, KEY 'key2' VALUE true)
--> '{"key1":1,"key2":true}'
```
In the second convention, you can omit the `KEY` keyword:
```sql theme={null}
SELECT json_object('key1' VALUE 1, 'key2' VALUE true)
--> '{"key1":1,"key2":true}'
```
### Argument types
The keys can be arbitrary expressions. They must be of character string type.
Each key is converted into a JSON text item, and it becomes a key in the
created JSON object. Keys must not be null.
The values can be arbitrary expressions. Each passed value is converted
into a JSON item according to its type, and optional `FORMAT` and
`ENCODING` specification.
You can pass SQL values of types boolean, numeric, and character string. They
are converted to corresponding JSON literals:
```sql theme={null}
SELECT json_object('x' : true, 'y' : 12e-1, 'z' : 'text')
--> '{"x":true,"y":1.2,"z":"text"}'
```
Additionally to SQL values, you can pass JSON values. They are character or
binary strings with a specified format and optional encoding:
```sql theme={null}
SELECT json_object(
'x' : '[ "text" ] ' FORMAT JSON,
'y' : X'5B0035005D00' FORMAT JSON ENCODING UTF16
)
--> '{"x":["text"],"y":[5]}'
```
You can also nest other JSON-returning functions. In that case, the `FORMAT`
option is implicit:
```sql theme={null}
SELECT json_object(
'x' : json_query('{"key" : [ "value" ]}', 'lax $.key')
)
--> '{"x":["value"]}'
```
Other passed values are cast to varchar, and they become JSON text literals:
```sql theme={null}
SELECT json_object(
'x' : DATE '2001-01-31',
'y' : UUID '12151fd2-7586-11e9-8f9e-2a86e4085a59'
)
--> '{"x":"2001-01-31","y":"12151fd2-7586-11e9-8f9e-2a86e4085a59"}'
```
You can omit the arguments altogether to get an empty object:
```sql theme={null}
SELECT json_object() --> '{}'
```
### Null handling
The values passed for JSON object keys must not be null. It is allowed to pass
`null` for JSON object values. A null value is treated according to the
specified null treatment option. If `NULL ON NULL` is specified, a JSON
object entry with `null` value is added to the result. If `ABSENT ON NULL`
is specified, the entry is omitted in the result. `NULL ON NULL` is the
default configuration.:
```sql theme={null}
SELECT json_object('x' : null, 'y' : 1)
--> '{"x":null,"y":1}'
SELECT json_object('x' : null, 'y' : 1 NULL ON NULL)
--> '{"x":null,"y":1}'
SELECT json_object('x' : null, 'y' : 1 ABSENT ON NULL)
--> '{"y":1}'
```
### Key uniqueness
If a duplicate key is encountered, it is handled according to the specified key
uniqueness constraint.
If `WITH UNIQUE KEYS` is specified, a duplicate key results in a query
failure:
```sql theme={null}
SELECT json_object('x' : null, 'x' : 1 WITH UNIQUE KEYS)
--> failure: "duplicate key passed to JSON_OBJECT function"
```
Note that this option is not supported if any of the arguments has a
`FORMAT` specification.
If `WITHOUT UNIQUE KEYS` is specified, duplicate keys are not supported due
to implementation limitation. `WITHOUT UNIQUE KEYS` is the default
configuration.
### Returned type
The SQL standard imposes that there is no dedicated data type to represent JSON
data in SQL. Instead, JSON data is represented as character or binary strings.
By default, the `json_object` function returns varchar containing the textual
representation of the JSON object. With the `RETURNING` clause, you can
specify other character string type:
```sql theme={null}
SELECT json_object('x' : 1 RETURNING VARCHAR(100))
--> '{"x":1}'
```
You can also specify to use varbinary and the required encoding as return type.
The default encoding is UTF8:
```sql theme={null}
SELECT json_object('x' : 1 RETURNING VARBINARY)
--> X'7b 22 78 22 3a 31 7d'
SELECT json_object('x' : 1 RETURNING VARBINARY FORMAT JSON ENCODING UTF8)
--> X'7b 22 78 22 3a 31 7d'
SELECT json_object('x' : 1 RETURNING VARBINARY FORMAT JSON ENCODING UTF16)
--> X'7b 00 22 00 78 00 22 00 3a 00 31 00 7d 00'
SELECT json_object('x' : 1 RETURNING VARBINARY FORMAT JSON ENCODING UTF32)
--> X'7b 00 00 00 22 00 00 00 78 00 00 00 22 00 00 00 3a 00 00 00 31 00 00 00 7d 00 00 00'
```
The following functions and operators are not compliant with the SQL
standard, and should be considered deprecated. According to the SQL
standard, there shall be no `JSON` data type. Instead, JSON values
should be represented as string values. The remaining functionality of the
following functions is covered by the functions described previously.
## Cast to JSON
The following types can be cast to JSON:
* `BOOLEAN`
* `TINYINT`
* `SMALLINT`
* `INTEGER`
* `BIGINT`
* `REAL`
* `DOUBLE`
* `VARCHAR`
Additionally, `ARRAY`, `MAP`, and `ROW` types can be cast to JSON when
the following requirements are met:
* `ARRAY` types can be cast when the element type of the array is one
of the supported types.
* `MAP` types can be cast when the key type of the map is `VARCHAR` and
the value type of the map is a supported type,
* `ROW` types can be cast when every field type of the row is a supported
type.
Cast operations with supported [character string types](/sql/reference/types#string) treat the input as a string, not validated as JSON.
This means that a cast operation with a string-type input of invalid JSON
results in a succesful cast to invalid JSON.
Instead, consider using the `json_parse` function to create validated JSON from a string.
The following examples show the behavior of casting to JSON with these types:
```sql theme={null}
SELECT CAST(NULL AS JSON);
-- NULL
SELECT CAST(1 AS JSON);
-- JSON '1'
SELECT CAST(9223372036854775807 AS JSON);
-- JSON '9223372036854775807'
SELECT CAST('abc' AS JSON);
-- JSON '"abc"'
SELECT CAST(true AS JSON);
-- JSON 'true'
SELECT CAST(1.234 AS JSON);
-- JSON '1.234'
SELECT CAST(ARRAY[1, 23, 456] AS JSON);
-- JSON '[1,23,456]'
SELECT CAST(ARRAY[1, NULL, 456] AS JSON);
-- JSON '[1,null,456]'
SELECT CAST(ARRAY[ARRAY[1, 23], ARRAY[456]] AS JSON);
-- JSON '[[1,23],[456]]'
SELECT CAST(MAP(ARRAY['k1', 'k2', 'k3'], ARRAY[1, 23, 456]) AS JSON);
-- JSON '{"k1":1,"k2":23,"k3":456}'
SELECT CAST(CAST(ROW(123, 'abc', true) AS
ROW(v1 BIGINT, v2 VARCHAR, v3 BOOLEAN)) AS JSON);
-- JSON '{"v1":123,"v2":"abc","v3":true}'
```
Casting from NULL to `JSON` is not straightforward. Casting
from a standalone `NULL` will produce SQL `NULL` instead of
`JSON 'null'`. However, when casting from arrays or map containing
`NULL`s, the produced `JSON` will have `null`s in it.
## Cast from JSON
Casting to `BOOLEAN`, `TINYINT`, `SMALLINT`, `INTEGER`,
`BIGINT`, `REAL`, `DOUBLE` or `VARCHAR` is supported.
Casting to `ARRAY` and `MAP` is supported when the element type of
the array is one of the supported types, or when the key type of the map
is `VARCHAR` and value type of the map is one of the supported types.
Behaviors of the casts are shown with the examples below:
```sql theme={null}
SELECT CAST(JSON 'null' AS VARCHAR);
-- NULL
SELECT CAST(JSON '1' AS INTEGER);
-- 1
SELECT CAST(JSON '9223372036854775807' AS BIGINT);
-- 9223372036854775807
SELECT CAST(JSON '"abc"' AS VARCHAR);
-- abc
SELECT CAST(JSON 'true' AS BOOLEAN);
-- true
SELECT CAST(JSON '1.234' AS DOUBLE);
-- 1.234
SELECT CAST(JSON '[1,23,456]' AS ARRAY(INTEGER));
-- [1, 23, 456]
SELECT CAST(JSON '[1,null,456]' AS ARRAY(INTEGER));
-- [1, NULL, 456]
SELECT CAST(JSON '[[1,23],[456]]' AS ARRAY(ARRAY(INTEGER)));
-- [[1, 23], [456]]
SELECT CAST(JSON '{"k1":1,"k2":23,"k3":456}' AS MAP(VARCHAR, INTEGER));
-- {k1=1, k2=23, k3=456}
SELECT CAST(JSON '{"v1":123,"v2":"abc","v3":true}' AS
ROW(v1 BIGINT, v2 VARCHAR, v3 BOOLEAN));
-- {v1=123, v2=abc, v3=true}
SELECT CAST(JSON '[123,"abc",true]' AS
ROW(v1 BIGINT, v2 VARCHAR, v3 BOOLEAN));
-- {v1=123, v2=abc, v3=true}
```
JSON arrays can have mixed element types and JSON maps can have mixed
value types. This makes it impossible to cast them to SQL arrays and maps in
some cases. To address this, Trino supports partial casting of arrays and maps:
```sql theme={null}
SELECT CAST(JSON '[[1, 23], 456]' AS ARRAY(JSON));
-- [JSON '[1,23]', JSON '456']
SELECT CAST(JSON '{"k1": [1, 23], "k2": 456}' AS MAP(VARCHAR, JSON));
-- {k1 = JSON '[1,23]', k2 = JSON '456'}
SELECT CAST(JSON '[null]' AS ARRAY(JSON));
-- [JSON 'null']
```
When casting from `JSON` to `ROW`, both JSON array and JSON object are supported.
## Other JSON functions
In addition to the functions explained in more details in the preceding
sections, the following functions are available:
### is\_json\_scalar
```
is_json_scalar(json) -> boolean
```
Determine if `json` is a scalar (i.e. a JSON number, a JSON string, `true`, `false` or `null`):
```sql theme={null}
SELECT is_json_scalar('1'); -- true
SELECT is_json_scalar('[1, 2, 3]'); -- false
```
### json\_array\_contains
```
json_array_contains(json, value) -> boolean
```
Determine if `value` exists in `json` (a string containing a JSON array):
```sql theme={null}
SELECT json_array_contains('[1, 2, 3]', 2); -- true
```
### json\_array\_get
```
json_array_get(json_array, index) -> json
```
The semantics of this function are broken. If the extracted element
is a string, it will be converted into an invalid `JSON` value that
is not properly quoted (the value will not be surrounded by quotes
and any interior quotes will not be escaped).
We recommend against using this function. It cannot be fixed without
impacting existing usages and may be removed in a future release.
Returns the element at the specified index into the `json_array`.
The index is zero-based:
```sql theme={null}
SELECT json_array_get('["a", [3, 9], "c"]', 0); -- JSON 'a' (invalid JSON)
SELECT json_array_get('["a", [3, 9], "c"]', 1); -- JSON '[3,9]'
```
This function also supports negative indexes for fetching element indexed
from the end of an array:
```sql theme={null}
SELECT json_array_get('["c", [3, 9], "a"]', -1); -- JSON 'a' (invalid JSON)
SELECT json_array_get('["c", [3, 9], "a"]', -2); -- JSON '[3,9]'
```
If the element at the specified index doesn't exist, the function returns null:
```sql theme={null}
SELECT json_array_get('[]', 0); -- NULL
SELECT json_array_get('["a", "b", "c"]', 10); -- NULL
SELECT json_array_get('["c", "b", "a"]', -10); -- NULL
```
### json\_array\_length
```
json_array_length(json) -> bigint
```
Returns the array length of `json` (a string containing a JSON array):
```sql theme={null}
SELECT json_array_length('[1, 2, 3]'); -- 3
```
### json\_extract
```
json_extract(json, json_path) -> json
```
Evaluates the [JSONPath]-like expression `json_path` on `json`
(a string containing JSON) and returns the result as a JSON string:
```sql theme={null}
SELECT json_extract(json, '$.store.book');
SELECT json_extract(json, '$.store[book]');
SELECT json_extract(json, '$.store["book name"]');
```
The [json\_query function](#json-query) provides a more powerful and
feature-rich alternative to parse and extract JSON data.
### json\_extract\_scalar
```
json_extract_scalar(json, json_path) -> varchar
```
Like [json\_extract](#json-extract), but returns the result value as a string (as opposed
to being encoded as JSON). The value referenced by `json_path` must be a
scalar (boolean, number or string).
```sql theme={null}
SELECT json_extract_scalar('[1, 2, 3]', '$[2]');
SELECT json_extract_scalar(json, '$.store.book[0].author');
```
### json\_format
```
json_format(json) -> varchar
```
Returns the JSON text serialized from the input JSON value.
This is inverse function to `json_parse`.
```sql theme={null}
SELECT json_format(JSON '[1, 2, 3]'); -- '[1,2,3]'
SELECT json_format(JSON '"a"'); -- '"a"'
```
[json\_format function](#json-format) and `CAST(json AS VARCHAR)` have completely
different semantics.
[json\_format function](#json-format) serializes the input JSON value to JSON text conforming to
`RFC 7159`. The JSON value can be a JSON object, a JSON array, a JSON string,
a JSON number, `true`, `false` or `null`.
```sql theme={null}
SELECT json_format(JSON '{"a": 1, "b": 2}'); -- '{"a":1,"b":2}'
SELECT json_format(JSON '[1, 2, 3]'); -- '[1,2,3]'
SELECT json_format(JSON '"abc"'); -- '"abc"'
SELECT json_format(JSON '42'); -- '42'
SELECT json_format(JSON 'true'); -- 'true'
SELECT json_format(JSON 'null'); -- 'null'
```
`CAST(json AS VARCHAR)` casts the JSON value to the corresponding SQL VARCHAR value.
For JSON string, JSON number, `true`, `false` or `null`, the cast
behavior is same as the corresponding SQL type. JSON object and JSON array
cannot be cast to VARCHAR.
```sql theme={null}
SELECT CAST(JSON '{"a": 1, "b": 2}' AS VARCHAR); -- ERROR!
SELECT CAST(JSON '[1, 2, 3]' AS VARCHAR); -- ERROR!
SELECT CAST(JSON '"abc"' AS VARCHAR); -- 'abc' (the double quote is gone)
SELECT CAST(JSON '42' AS VARCHAR); -- '42'
SELECT CAST(JSON 'true' AS VARCHAR); -- 'true'
SELECT CAST(JSON 'null' AS VARCHAR); -- NULL
```
### json\_parse
```
json_parse(string) -> json
```
Returns the JSON value deserialized from the input JSON text.
This is inverse function to [json\_format function](#json-format):
```
SELECT json_parse('[1, 2, 3]'); -- JSON '[1,2,3]'
SELECT json_parse('"abc"'); -- JSON '"abc"'
```
`json_parse` and `CAST(string AS JSON)` have completely
different semantics.
`json_parse` expects a JSON text conforming to `RFC 7159`, and returns
the JSON value deserialized from the JSON text.
The JSON value can be a JSON object, a JSON array, a JSON string, a JSON number,
`true`, `false` or `null`.
```sql theme={null}
SELECT json_parse('not_json'); -- ERROR!
SELECT json_parse('["a": 1, "b": 2]'); -- JSON '["a": 1, "b": 2]'
SELECT json_parse('[1, 2, 3]'); -- JSON '[1,2,3]'
SELECT json_parse('"abc"'); -- JSON '"abc"'
SELECT json_parse('42'); -- JSON '42'
SELECT json_parse('true'); -- JSON 'true'
SELECT json_parse('null'); -- JSON 'null'
```
`CAST(string AS JSON)` takes any VARCHAR value as input, and returns
a JSON string with its value set to input string.
```sql theme={null}
SELECT CAST('not_json' AS JSON); -- JSON '"not_json"'
SELECT CAST('["a": 1, "b": 2]' AS JSON); -- JSON '"[\"a\": 1, \"b\": 2]"'
SELECT CAST('[1, 2, 3]' AS JSON); -- JSON '"[1, 2, 3]"'
SELECT CAST('"abc"' AS JSON); -- JSON '"\"abc\""'
SELECT CAST('42' AS JSON); -- JSON '"42"'
SELECT CAST('true' AS JSON); -- JSON '"true"'
SELECT CAST('null' AS JSON); -- JSON '"null"'
```
### json\_size
```
json_size(json, json_path) -> bigint
```
Like [json\_extract](#json-extract), but returns the size of the value.
For objects or arrays, the size is the number of members,
and the size of a scalar value is zero.
```
SELECT json_size('{"x": {"a": 1, "b": 2}}', '$.x'); -- 2
SELECT json_size('{"x": [1, 2, 3]}', '$.x'); -- 3
SELECT json_size('{"x": {"a": 1, "b": 2}}', '$.x.a'); -- 0
```
[jsonpath]: http://goessner.net/articles/JsonPath/
# Lambda expressions
Source: https://docs.upsolve.ai/sql/functions/lambda
Lambda expressions are anonymous functions which are passed as
arguments to higher-order SQL functions.
Lambda expressions are written with `->`:
```sql theme={null}
x -> x + 1
(x, y) -> x + y
x -> regexp_like(x, 'a+')
x -> x[1] / x[2]
x -> IF(x > 0, x, -x)
x -> COALESCE(x, 0)
x -> CAST(x AS JSON)
x -> x + TRY(1 / 0)
```
## Limitations
Most SQL expressions can be used in a lambda body, with a few exceptions:
* Subqueries are not supported: `x -> 2 + (SELECT 3)`
* Aggregations are not supported: `x -> max(y)`
## Examples
Obtain the squared elements of an array column with [transform()](/sql/functions/array#transform):
```sql theme={null}
SELECT numbers,
transform(numbers, n -> n * n) as squared_numbers
FROM (
VALUES
(ARRAY[1, 2]),
(ARRAY[3, 4]),
(ARRAY[5, 6, 7])
) AS t(numbers);
```
```
numbers | squared_numbers
-----------+-----------------
[1, 2] | [1, 4]
[3, 4] | [9, 16]
[5, 6, 7] | [25, 36, 49]
(3 rows)
```
The function [transform()](/sql/functions/array#transform) can be also employed to safely cast the elements
of an array to strings:
```sql theme={null}
SELECT transform(prices, n -> TRY_CAST(n AS VARCHAR) || '$') as price_tags
FROM (
VALUES
(ARRAY[100, 200]),
(ARRAY[30, 4])
) AS t(prices);
```
```
price_tags
--------------
[100$, 200$]
[30$, 4$]
(2 rows)
```
Besides the array column being manipulated,
other columns can be captured as well within the lambda expression.
The following statement provides a showcase of this feature
for calculating the value of the linear function `f(x) = ax + b`
with [transform()](/sql/functions/array#transform):
```sql theme={null}
SELECT xvalues,
a,
b,
transform(xvalues, x -> a * x + b) as linear_function_values
FROM (
VALUES
(ARRAY[1, 2], 10, 5),
(ARRAY[3, 4], 4, 2)
) AS t(xvalues, a, b);
```
```
xvalues | a | b | linear_function_values
---------+----+---+------------------------
[1, 2] | 10 | 5 | [15, 25]
[3, 4] | 4 | 2 | [14, 18]
(2 rows)
```
Find the array elements containing at least one value greater than `100`
with [any\_match()](/sql/functions/array#any-match):
```sql theme={null}
SELECT numbers
FROM (
VALUES
(ARRAY[1,NULL,3]),
(ARRAY[10,20,30]),
(ARRAY[100,200,300])
) AS t(numbers)
WHERE any_match(numbers, n -> COALESCE(n, 0) > 100);
-- [100, 200, 300]
```
Capitalize the first word in a string via [regexp\_replace()](/sql/functions/regexp#regexp-replace):
```sql theme={null}
SELECT regexp_replace('once upon a time ...', '^(\w)(\w*)(\s+.*)$',x -> upper(x[1]) || x[2] || x[3]);
-- Once upon a time ...
```
Lambda expressions can be also applied in aggregation functions.
Following statement is a sample the overly complex calculation of the sum of all elements of a column
by making use of [reduce\_agg()](/sql/functions/aggregate#reduce-agg):
```sql theme={null}
SELECT reduce_agg(value, 0, (a, b) -> a + b, (a, b) -> a + b) sum_values
FROM (
VALUES (1), (2), (3), (4), (5)
) AS t(value);
-- 15
```
# Map
Source: https://docs.upsolve.ai/sql/functions/map
# Map functions and operators
## Subscript operator: \[]
The `[]` operator is used to retrieve the value corresponding to a given key from a map:
```sql theme={null}
SELECT name_to_age_map['Bob'] AS bob_age;
```
## Map functions
### cardinality
```
cardinality(x) -> bigint
```
Returns the cardinality (size) of the map `x`.
### element\_at
```
element_at(map(K,V), key) -> V
```
Returns value for given `key`, or `NULL` if the key is not contained in the map.
### map
```
map() -> map
```
Returns an empty map.
```sql theme={null}
SELECT map();
-- {}
```
```
map(array(K), array(V)) -> map(K,V)
```
Returns a map created using the given key/value arrays.
```sql theme={null}
SELECT map(ARRAY[1,3], ARRAY[2,4]);
-- {1 -> 2, 3 -> 4}
```
See also [map\_agg()](/sql/functions/aggregate#map-agg) and [multimap\_agg()](/sql/functions/aggregate#multimap-agg) for creating a map as an aggregation.
### map\_from\_entries
```
map_from_entries(array(row(K,V))) -> map(K,V)
```
Returns a map created from the given array of entries.
```sql theme={null}
SELECT map_from_entries(ARRAY[(1, 'x'), (2, 'y')]);
-- {1 -> 'x', 2 -> 'y'}
```
### multimap\_from\_entries
```
multimap_from_entries(array(row(K,V))) -> map(K,array(V))
```
Returns a multimap created from the given array of entries. Each key can be associated with multiple values.
```sql theme={null}
SELECT multimap_from_entries(ARRAY[(1, 'x'), (2, 'y'), (1, 'z')]);
-- {1 -> ['x', 'z'], 2 -> ['y']}
```
### map\_entries
```
map_entries(map(K,V)) -> array(row(K,V))
```
Returns an array of all entries in the given map.
```sql theme={null}
SELECT map_entries(MAP(ARRAY[1, 2], ARRAY['x', 'y']));
-- [ROW(1, 'x'), ROW(2, 'y')]
```
### map\_concat
```
map_concat(map1(K,V), map2(K,V), ..., mapN(K,V)) -> map(K,V)
```
Returns the union of all the given maps. If a key is found in multiple given maps,
that key's value in the resulting map comes from the last one of those maps.
### map\_filter
```
map_filter(map(K,V), function(K,V,boolean)) -> map(K,V)
```
Constructs a map from those entries of `map` for which `function` returns true:
```sql theme={null}
SELECT map_filter(MAP(ARRAY[], ARRAY[]), (k, v) -> true);
-- {}
SELECT map_filter(MAP(ARRAY[10, 20, 30], ARRAY['a', NULL, 'c']),
(k, v) -> v IS NOT NULL);
-- {10 -> a, 30 -> c}
SELECT map_filter(MAP(ARRAY['k1', 'k2', 'k3'], ARRAY[20, 3, 15]),
(k, v) -> v > 10);
-- {k1 -> 20, k3 -> 15}
```
### map\_keys
```
map_keys(x(K,V)) -> array(K)
```
Returns all the keys in the map `x`.
### map\_values
```
map_values(x(K,V)) -> array(V)
```
Returns all the values in the map `x`.
### map\_zip\_with
```
map_zip_with(map(K,V1), map(K,V2), function(K,V1,V2,V3)) -> map(K,V3)
```
Merges the two given maps into a single map by applying `function` to the pair of values with the same key.
For keys only presented in one map, NULL will be passed as the value for the missing key.
```sql theme={null}
SELECT map_zip_with(MAP(ARRAY[1, 2, 3], ARRAY['a', 'b', 'c']),
MAP(ARRAY[1, 2, 3], ARRAY['d', 'e', 'f']),
(k, v1, v2) -> concat(v1, v2));
-- {1 -> ad, 2 -> be, 3 -> cf}
SELECT map_zip_with(MAP(ARRAY['k1', 'k2'], ARRAY[1, 2]),
MAP(ARRAY['k2', 'k3'], ARRAY[4, 9]),
(k, v1, v2) -> (v1, v2));
-- {k1 -> ROW(1, null), k2 -> ROW(2, 4), k3 -> ROW(null, 9)}
SELECT map_zip_with(MAP(ARRAY['a', 'b', 'c'], ARRAY[1, 8, 27]),
MAP(ARRAY['a', 'b', 'c'], ARRAY[1, 2, 3]),
(k, v1, v2) -> k || CAST(v1 / v2 AS VARCHAR));
-- {a -> a1, b -> b4, c -> c9}
```
### transform\_keys
```
transform_keys(map(K1,V), function(K1,V,K2)) -> map(K2,V)
```
Returns a map that applies `function` to each entry of `map` and transforms the keys:
```sql theme={null}
SELECT transform_keys(MAP(ARRAY[], ARRAY[]), (k, v) -> k + 1);
-- {}
SELECT transform_keys(MAP(ARRAY [1, 2, 3], ARRAY ['a', 'b', 'c']),
(k, v) -> k + 1);
-- {2 -> a, 3 -> b, 4 -> c}
SELECT transform_keys(MAP(ARRAY ['a', 'b', 'c'], ARRAY [1, 2, 3]),
(k, v) -> v * v);
-- {1 -> 1, 4 -> 2, 9 -> 3}
SELECT transform_keys(MAP(ARRAY ['a', 'b'], ARRAY [1, 2]),
(k, v) -> k || CAST(v as VARCHAR));
-- {a1 -> 1, b2 -> 2}
SELECT transform_keys(MAP(ARRAY [1, 2], ARRAY [1.0, 1.4]),
(k, v) -> MAP(ARRAY[1, 2], ARRAY['one', 'two'])[k]);
-- {one -> 1.0, two -> 1.4}
```
### transform\_values
```
transform_values(map(K,V1), function(K,V1,V2)) -> map(K,V2)
```
Returns a map that applies `function` to each entry of `map` and transforms the values:
```sql theme={null}
SELECT transform_values(MAP(ARRAY[], ARRAY[]), (k, v) -> v + 1);
-- {}
SELECT transform_values(MAP(ARRAY [1, 2, 3], ARRAY [10, 20, 30]),
(k, v) -> v + k);
-- {1 -> 11, 2 -> 22, 3 -> 33}
SELECT transform_values(MAP(ARRAY [1, 2, 3], ARRAY ['a', 'b', 'c']),
(k, v) -> k * k);
-- {1 -> 1, 2 -> 4, 3 -> 9}
SELECT transform_values(MAP(ARRAY ['a', 'b'], ARRAY [1, 2]),
(k, v) -> k || CAST(v as VARCHAR));
-- {a -> a1, b -> b2}
SELECT transform_values(MAP(ARRAY [1, 2], ARRAY [1.0, 1.4]),
(k, v) -> MAP(ARRAY[1, 2], ARRAY['one', 'two'])[k]
|| '_' || CAST(v AS VARCHAR));
-- {1 -> one_1.0, 2 -> two_1.4}
```
# Machine learning functions
Source: https://docs.upsolve.ai/sql/functions/ml
The machine learning plugin provides machine learning functionality
as an aggregation function. It enables you to train Support Vector Machine (SVM)
based classifiers and regressors for the supervised learning problems.
The machine learning functions are not optimized for distributed processing.
The capability to train large data sets is limited by this execution of the
final training on a single instance.
## Feature vector
To solve a problem with the machine learning technique, especially as a
supervised learning problem, it is necessary to represent the data set
with the sequence of pairs of labels and feature vector. A label is a
target value you want to predict from the unseen feature and a feature is a
A N-dimensional vector whose elements are numerical values. In Trino, a
feature vector is represented as a map-type value, whose key is an index
of each feature, so that it can express a sparse vector.
Since classifiers and regressors can recognize the map-type feature
vector, there is a function to construct the feature from the existing
numerical values, [features()](#features):
```sql theme={null}
SELECT features(1.0, 2.0, 3.0) AS features;
```
```text theme={null}
features
-----------------------
{0=1.0, 1=2.0, 2=3.0}
```
The output from [features()](#features) can be directly passed to ML functions.
## Classification
Classification is a type of supervised learning problem to predict the distinct
label from the given feature vector. The interface looks similar to the
construction of the SVM model from the sequence of pairs of labels and features
implemented in Teradata Aster or [BigQuery ML](https://cloud.google.com/bigquery-ml/docs/bigqueryml-intro).
The function to train a classification model looks like as follows:
```sql theme={null}
SELECT
learn_classifier(
species,
features(sepal_length, sepal_width, petal_length, petal_width)
) AS model
FROM
iris
```
It returns the trained model in a serialized format.
```text theme={null}
model
-------------------------------------------------
3c 43 6c 61 73 73 69 66 69 65 72 28 76 61 72 63
68 61 72 29 3e
```
[classify()](#classify) returns the predicted label by using the trained model.
The trained model can not be saved natively, and needs to be passed in
the format of a nested query:
```sql theme={null}
SELECT
classify(features(5.9, 3, 5.1, 1.8), model) AS predicted_label
FROM (
SELECT
learn_classifier(species, features(sepal_length, sepal_width, petal_length, petal_width)) AS model
FROM
iris
) t
```
```text theme={null}
predicted_label
-----------------
Iris-virginica
```
As a result you need to run the training process at the same time when predicting values.
Internally, the model is trained by [libsvm](https://www.csie.ntu.edu.tw/~cjlin/libsvm/).
You can use [learn\_libsvm\_classifier()](#learn-libsvm-classifier) to control the internal parameters of the model.
## Regression
Regression is another type of supervised learning problem, predicting continuous
value, unlike the classification problem. The target must be numerical values that can
be described as `double`.
The following code shows the creation of the model predicting `sepal_length`
from the other 3 features:
```sql theme={null}
SELECT
learn_regressor(sepal_length, features(sepal_width, petal_length, petal_width)) AS model
FROM
iris
```
The way to use the model is similar to the classification case:
```sql theme={null}
SELECT
regress(features(3, 5.1, 1.8), model) AS predicted_target
FROM (
SELECT
learn_regressor(sepal_length, features(sepal_width, petal_length, petal_width)) AS model
FROM iris
) t;
```
```text theme={null}
predicted_target
-------------------
6.407376822560477
```
Internally, the model is trained by [libsvm](https://www.csie.ntu.edu.tw/~cjlin/libsvm/).
[learn\_libsvm\_regressor()](#learn-libsvm-regressor) provides you a way to control the training process.
## Machine learning functions
### features
```
features(double, ...) -> map(bigint, double)
```
Returns the map representing the feature vector.
### learn\_classifier
```
learn_classifier(label, features) -> Classifier
```
Returns an SVM-based classifier model, trained with the given label and feature data sets.
### learn\_libsvm\_classifier
```
learn_libsvm_classifier(label, features, params) -> Classifier
```
Returns an SVM-based classifier model, trained with the given label and feature data sets.
You can control the training process by libsvm parameters.
### classify
```
classify(features, model) -> label
```
Returns a label predicted by the given classifier SVM model.
### learn\_regressor
```
learn_regressor(target, features) -> Regressor
```
Returns an SVM-based regressor model, trained with the given target and feature data sets.
### learn\_libsvm\_regressor
```
learn_libsvm_regressor(target, features, params) -> Regressor
```
Returns an SVM-based regressor model, trained with the given target and feature data sets.
You can control the training process by libsvm parameters.
### regress
```
regress(features, model) -> target
```
Returns a predicted target value by the given regressor SVM model.
# Quantile digest functions
Source: https://docs.upsolve.ai/sql/functions/qdigest
## Data structures
A quantile digest is a data sketch which stores approximate percentile
information. The Trino type for this data structure is called `qdigest`,
and it takes a parameter which must be one of `bigint`, `double` or
`real` which represent the set of numbers that may be ingested by the
`qdigest`. They may be merged without losing precision, and for storage
and retrieval they may be cast to/from `VARBINARY`.
## Functions
### merge
```
merge(qdigest) -> qdigest
```
Merges all input `qdigest`s into a single `qdigest`.
### value\_at\_quantile
```
value_at_quantile(qdigest(T), quantile) -> T
```
Returns the approximate percentile value from the quantile digest given
the number `quantile` between 0 and 1.
### quantile\_at\_value
```
quantile_at_value(qdigest(T), T) -> quantile
```
Returns the approximate `quantile` number between 0 and 1 from the
quantile digest given an input value. Null is returned if the quantile digest
is empty or the input value is outside of the range of the quantile digest.
### values\_at\_quantiles
```
values_at_quantiles(qdigest(T), quantiles) -> array(T)
```
Returns the approximate percentile values as an array given the input
quantile digest and array of values between 0 and 1 which
represent the quantiles to return.
### qdigest\_agg
```
qdigest_agg(x) -> qdigest([same as x])
```
Returns the `qdigest` which is composed of all input values of `x`.
```
qdigest_agg(x, w) -> qdigest([same as x])
```
Returns the `qdigest` which is composed of all input values of `x` using
the per-item weight `w`.
```
qdigest_agg(x, w, accuracy) -> qdigest([same as x])
```
Returns the `qdigest` which is composed of all input values of `x` using
the per-item weight `w` and maximum error of `accuracy`. `accuracy`
must be a value greater than zero and less than one, and it must be constant
for all input rows.
# Regexp
Source: https://docs.upsolve.ai/sql/functions/regexp
All of the regular expression functions use the [java pattern] syntax,
with a few notable exceptions:
* When using multi-line mode (enabled via the `(?m)` flag),
only `\n` is recognized as a line terminator. Additionally,
the `(?d)` flag is not supported and must not be used.
* Case-insensitive matching (enabled via the `(?i)` flag) is always
performed in a Unicode-aware manner. However, context-sensitive and
local-sensitive matching is not supported. Additionally, the
`(?u)` flag is not supported and must not be used.
* Surrogate pairs are not supported. For example, `\uD800\uDC00` is
not treated as `U+10000` and must be specified as `\x{10000}`.
* Boundaries (`\b`) are incorrectly handled for a non-spacing mark
without a base character.
* `\Q` and `\E` are not supported in character classes
(such as `[A-Z123]`) and are instead treated as literals.
* Unicode character classes (`\p{prop}`) are supported with
the following differences:
* All underscores in names must be removed. For example, use
`OldItalic` instead of `Old_Italic`.
* Scripts must be specified directly, without the
`Is`, `script=` or `sc=` prefixes.
Example: `\p{Hiragana}`
* Blocks must be specified with the `In` prefix.
The `block=` and `blk=` prefixes are not supported.
Example: `\p{Mongolian}`
* Categories must be specified directly, without the `Is`,
`general_category=` or `gc=` prefixes.
Example: `\p{L}`
* Binary properties must be specified directly, without the `Is`.
Example: `\p{NoncharacterCodePoint}`
### regexp\_count
```
regexp_count(string, pattern) -> bigint
```
Returns the number of occurrence of `pattern` in `string`:
```sql theme={null}
SELECT regexp_count('1a 2b 14m', '\s*[a-z]+\s*'); -- 3
```
### regexp\_extract\_all
```
regexp_extract_all(string, pattern) -> array(varchar)
```
Returns the substring(s) matched by the regular expression `pattern`
in `string`:
```sql theme={null}
SELECT regexp_extract_all('1a 2b 14m', '\d+'); -- [1, 2, 14]
```
```
regexp_extract_all(string, pattern, group) -> array(varchar)
```
Finds all occurrences of the regular expression `pattern` in `string`
and returns the [capturing group number] `group`:
```sql theme={null}
SELECT regexp_extract_all('1a 2b 14m', '(\d+)([a-z]+)', 2); -- ['a', 'b', 'm']
```
### regexp\_extract
```
regexp_extract(string, pattern) -> varchar
```
Returns the first substring matched by the regular expression `pattern`
in `string`:
```sql theme={null}
SELECT regexp_extract('1a 2b 14m', '\d+'); -- 1
```
```
regexp_extract(string, pattern, group) -> varchar
```
Finds the first occurrence of the regular expression `pattern` in
`string` and returns the [capturing group number] `group`:
```sql theme={null}
SELECT regexp_extract('1a 2b 14m', '(\d+)([a-z]+)', 2); -- 'a'
```
### regexp\_like
```
regexp_like(string, pattern) -> boolean
```
Evaluates the regular expression `pattern` and determines if it is
contained within `string`.
The `pattern` only needs to be contained within
`string`, rather than needing to match all of `string`. In other words,
this performs a *contains* operation rather than a *match* operation. You can
match the entire string by anchoring the pattern using `^` and `$`:
```sql theme={null}
SELECT regexp_like('1a 2b 14m', '\d+b'); -- true
```
### regexp\_position
```
regexp_position(string, pattern) -> integer
```
Returns the index of the first occurrence (counting from 1) of `pattern` in `string`.
Returns -1 if not found:
```sql theme={null}
SELECT regexp_position('I have 23 apples, 5 pears and 13 oranges', '\b\d+\b'); -- 8
```
### regexp\_position
```
regexp_position(string, pattern, start) -> integer
```
Returns the index of the first occurrence of `pattern` in `string`,
starting from `start` (include `start`). Returns -1 if not found:
```sql theme={null}
SELECT regexp_position('I have 23 apples, 5 pears and 13 oranges', '\b\d+\b', 5); -- 8
SELECT regexp_position('I have 23 apples, 5 pears and 13 oranges', '\b\d+\b', 12); -- 19
```
```
regexp_position(string, pattern, start, occurrence) -> integer
```
Returns the index of the nth `occurrence` of `pattern` in `string`,
starting from `start` (include `start`). Returns -1 if not found:
```sql theme={null}
SELECT regexp_position('I have 23 apples, 5 pears and 13 oranges', '\b\d+\b', 12, 1); -- 19
SELECT regexp_position('I have 23 apples, 5 pears and 13 oranges', '\b\d+\b', 12, 2); -- 31
SELECT regexp_position('I have 23 apples, 5 pears and 13 oranges', '\b\d+\b', 12, 3); -- -1
```
### regexp\_replace
```
regexp_replace(string, pattern) -> varchar
```
Removes every instance of the substring matched by the regular expression
`pattern` from `string`:
```sql theme={null}
SELECT regexp_replace('1a 2b 14m', '\d+[ab] '); -- '14m'
```
```
regexp_replace(string, pattern, replacement) -> varchar
```
Replaces every instance of the substring matched by the regular expression
`pattern` in `string` with `replacement`. [Capturing groups] can be
referenced in `replacement` using `$g` for a numbered group or
`${name}` for a named group. A dollar sign (`$`) may be included in the
replacement by escaping it with a backslash (`\$`):
```sql theme={null}
SELECT regexp_replace('1a 2b 14m', '(\d+)([ab]) ', '3c$2 '); -- '3ca 3cb 14m'
```
```
regexp_replace(string, pattern, function) -> varchar
```
Replaces every instance of the substring matched by the regular expression
`pattern` in `string` using `function`. The [lambda expression](/sql/functions/lambda)
`function` is invoked for each match with the [capturing groups] passed as an
array. Capturing group numbers start at one; there is no group for the entire match
(if you need this, surround the entire expression with parenthesis).
```sql theme={null}
SELECT regexp_replace('new york', '(\w)(\w*)', x -> upper(x[1]) || lower(x[2])); --'New York'
```
### regexp\_split
```
regexp_split(string, pattern) -> array(varchar)
```
Splits `string` using the regular expression `pattern` and returns an
array. Trailing empty strings are preserved:
```sql theme={null}
SELECT regexp_split('1a 2b 14m', '\s*[a-z]+\s*'); -- [1, 2, 14, ]
```
[capturing group number]: https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/util/regex/Pattern.html#gnumber
[capturing groups]: https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/util/regex/Pattern.html#cg
[java pattern]: https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/util/regex/Pattern.html
# Set Digest functions
Source: https://docs.upsolve.ai/sql/functions/setdigest
Trino offers several functions that deal with the
[MinHash](https://wikipedia.org/wiki/MinHash) technique.
MinHash is used to quickly estimate the
[Jaccard similarity coefficient](https://wikipedia.org/wiki/Jaccard_index)
between two sets.
It is commonly used in data mining to detect near-duplicate web pages at scale.
By using this information, the search engines efficiently avoid showing
within the search results two pages that are nearly identical.
The following example showcases how the Set Digest functions can be
used to naively estimate the similarity between texts. The input texts
are split by using the function [ngrams()](/sql/functions/array#ngrams) to
[4-shingles](https://wikipedia.org/wiki/W-shingling) which are
used as input for creating a set digest of each initial text.
The set digests are compared to each other to get an
approximation of the similarity of their corresponding
initial texts:
```sql theme={null}
WITH text_input(id, text) AS (
VALUES
(1, 'The quick brown fox jumps over the lazy dog'),
(2, 'The quick and the lazy'),
(3, 'The quick brown fox jumps over the dog')
),
text_ngrams(id, ngrams) AS (
SELECT id,
transform(
ngrams(
split(text, ' '),
4
),
token -> array_join(token, ' ')
)
FROM text_input
),
minhash_digest(id, digest) AS (
SELECT id,
(SELECT make_set_digest(v) FROM unnest(ngrams) u(v))
FROM text_ngrams
),
setdigest_side_by_side(id1, digest1, id2, digest2) AS (
SELECT m1.id as id1,
m1.digest as digest1,
m2.id as id2,
m2.digest as digest2
FROM (SELECT id, digest FROM minhash_digest) m1
JOIN (SELECT id, digest FROM minhash_digest) m2
ON m1.id != m2.id AND m1.id < m2.id
)
SELECT id1,
id2,
intersection_cardinality(digest1, digest2) AS intersection_cardinality,
jaccard_index(digest1, digest2) AS jaccard_index
FROM setdigest_side_by_side
ORDER BY id1, id2;
```
```text theme={null}
id1 | id2 | intersection_cardinality | jaccard_index
-----+-----+--------------------------+---------------
1 | 2 | 0 | 0.0
1 | 3 | 4 | 0.6
2 | 3 | 0 | 0.0
```
The above result listing points out, as expected, that the texts
with the id `1` and `3` are quite similar.
One may argue that the text with the id `2` is somewhat similar to
the texts with the id `1` and `3`. Due to the fact in the example above
*4-shingles* are taken into account for measuring the similarity of the texts,
there are no intersections found for the text pairs `1` and `2`, respectively
`3` and `2` and therefore there the similarity index for these text pairs
is `0`.
## Data structures
Trino implements Set Digest data sketches by encapsulating the following components:
* [HyperLogLog](https://wikipedia.org/wiki/HyperLogLog)
* [MinHash with a single hash function](http://wikipedia.org/wiki/MinHash#Variant_with_a_single_hash_function)
The HyperLogLog structure is used for the approximation of the distinct elements
in the original set.
The MinHash structure is used to store a low memory footprint signature of the original set.
The similarity of any two sets is estimated by comparing their signatures.
The Trino type for this data structure is called `setdigest`.
Trino offers the ability to merge multiple Set Digest data sketches.
## Serialization
Data sketches can be serialized to and deserialized from `varbinary`. This
allows them to be stored for later use.
## Functions
### make\_set\_digest
```
make_set_digest(x) -> setdigest
Composes all input values of `x` into a `setdigest`.
```
Create a `setdigest` corresponding to a `bigint` array:
```sql theme={null}
SELECT make_set_digest(value)
FROM (VALUES 1, 2, 3) T(value);
```
Create a `setdigest` corresponding to a `varchar` array:
```sql theme={null}
SELECT make_set_digest(value)
FROM (VALUES 'Trino', 'SQL', 'on', 'everything') T(value);
```
### merge\_set\_digest
```
merge_set_digest(setdigest) -> setdigest
```
Returns the `setdigest` of the aggregate union of the individual `setdigest`
Set Digest structures.
### cardinality
```
cardinality(setdigest) -> long
```
Returns the cardinality of the set digest from its internal
`HyperLogLog` component.
Examples:
```sql theme={null}
SELECT cardinality(make_set_digest(value))
FROM (VALUES 1, 2, 2, 3, 3, 3, 4, 4, 4, 4, 5) T(value);
-- 5
```
### intersection\_cardinality
```
intersection_cardinality(x,y) -> long
Returns the estimation for the cardinality of the intersection of the two set digests.
`x` and `y` must be of type `setdigest`
```
Examples:
```sql theme={null}
SELECT intersection_cardinality(make_set_digest(v1), make_set_digest(v2))
FROM (VALUES (1, 1), (NULL, 2), (2, 3), (3, 4)) T(v1, v2);
-- 3
```
### jaccard\_index
```
jaccard_index(x, y) -> double
```
Returns the estimation of [Jaccard index](https://wikipedia.org/wiki/Jaccard_index) for
the two set digests.
`x` and `y` must be of type `setdigest`.
Examples:
```sql theme={null}
SELECT jaccard_index(make_set_digest(v1), make_set_digest(v2))
FROM (VALUES (1, 1), (NULL,2), (2, 3), (NULL, 4)) T(v1, v2);
-- 0.5
```
### hash\_counts
```
hash_counts(x) -> map(bigint, smallint)
```
Returns a map containing the [Murmur3Hash128](https://wikipedia.org/wiki/MurmurHash#MurmurHash3)
hashed values and the count of their occurences within
the internal `MinHash` structure belonging to `x`.
`x` must be of type `setdigest`.
Examples:
```sql theme={null}
SELECT hash_counts(make_set_digest(value))
FROM (VALUES 1, 1, 1, 2, 2) T(value);
-- {19144387141682250=3, -2447670524089286488=2}
```
# String functions and operators
Source: https://docs.upsolve.ai/sql/functions/string
## String operators
The `||` operator performs concatenation.
The `LIKE` statement can be used for pattern matching and is documented in
[Pattern Comparison: LIKE](/sql/functions/comparison#pattern-comparison-like)
## String functions
These functions assume that the input strings contain valid UTF-8 encoded
Unicode code points. There are no explicit checks for valid UTF-8 and
the functions may return incorrect results on invalid UTF-8.
Invalid UTF-8 data can be corrected with [from\_utf8()](#from-utf8).
Additionally, the functions operate on Unicode code points and not user
visible *characters* (or *grapheme clusters*). Some languages combine
multiple code points into a single user-perceived *character*, the basic
unit of a writing system for a language, but the functions will treat each
code point as a separate unit.
The [lower()](#lower) and [upper()](#upper) functions do not perform
locale-sensitive, context-sensitive, or one-to-many mappings required for
some languages. Specifically, this will return incorrect results for
Lithuanian, Turkish and Azeri.
### chr
```
chr(n) -> varchar
```
Returns the Unicode code point `n` as a single character string.
### codepoint
```
codepoint(string) -> integer
```
Returns the Unicode code point of the only character of `string`.
### concat
```
concat(string1, ..., stringN) -> varchar
```
Returns the concatenation of `string1`, `string2`, `...`, `stringN`.
This function provides the same functionality as the
SQL-standard concatenation operator (`||`).
### concat\_ws
```
concat_ws(string0, string1, ..., stringN) -> varchar
```
Returns the concatenation of `string1`, `string2`, `...`, `stringN`
using `string0` as a separator. If `string0` is null, then the return
value is null. Any null values provided in the arguments after the
separator are skipped.
```
concat_ws(string0, array(varchar)) -> varchar
```
Returns the concatenation of elements in the array using `string0` as a
separator. If `string0` is null, then the return value is null. Any
null values in the array are skipped.
### format
```
format(format, args...) -> varchar
```
See [format()](/sql/functions/conversion#format).
### hamming\_distance
```
hamming_distance(string1, string2) -> bigint
```
Returns the Hamming distance of `string1` and `string2`,
i.e. the number of positions at which the corresponding characters are different.
Note that the two strings must have the same length.
### length
```
length(string) -> bigint
```
Returns the length of `string` in characters.
### levenshtein\_distance
```
levenshtein_distance(string1, string2) -> bigint
```
Returns the Levenshtein edit distance of `string1` and `string2`,
i.e. the minimum number of single-character edits (insertions,
deletions or substitutions) needed to change `string1` into `string2`.
### lower
```
lower(string) -> varchar
```
Converts `string` to lowercase.
### lpad
```
lpad(string, size, padstring) -> varchar
```
Left pads `string` to `size` characters with `padstring`.
If `size` is less than the length of `string`, the result is
truncated to `size` characters. `size` must not be negative
and `padstring` must be non-empty.
### ltrim
```
ltrim(string) -> varchar
```
Removes leading whitespace from `string`.
### luhn\_check
```
luhn_check(string) -> boolean
```
Tests whether a `string` of digits is valid according to the
[Luhn algorithm](https://wikipedia.org/wiki/Luhn_algorithm).
This checksum function, also known as `modulo 10` or `mod 10`, is
widely applied on credit card numbers and government identification numbers
to distinguish valid numbers from mistyped, incorrect numbers.
Valid identification number:
```sql theme={null}
select luhn_check('79927398713');
-- true
```
Invalid identification number:
```sql theme={null}
select luhn_check('79927398714');
-- false
```
### position
```
position(substring IN string) -> bigint
```
Returns the starting position of the first instance of `substring` in
`string`. Positions start with `1`. If not found, `0` is returned.
This SQL-standard function has special syntax and uses the
`IN` keyword for the arguments. See also [strpos()](#strpos).
### replace
```
replace(string, search) -> varchar
```
Removes all instances of `search` from `string`.
```
replace(string, search, replace) -> varchar
```
Replaces all instances of `search` with `replace` in `string`.
### reverse
```
reverse(string) -> varchar
```
Returns `string` with the characters in reverse order.
### rpad
```
rpad(string, size, padstring) -> varchar
```
Right pads `string` to `size` characters with `padstring`.
If `size` is less than the length of `string`, the result is
truncated to `size` characters. `size` must not be negative
and `padstring` must be non-empty.
### rtrim
```
rtrim(string) -> varchar
```
Removes trailing whitespace from `string`.
### soundex
```
soundex(char) -> string
```
`soundex` returns a character string containing the phonetic representation of `char`.
: It is typically used to evaluate the similarity of two expressions phonetically, that is
how the string sounds when spoken:
```sql theme={null}
SELECT name
FROM nation
WHERE SOUNDEX(name) = SOUNDEX('CHYNA');
name |
-------+----
CHINA |
(1 row)
```
### split
```
split(string, delimiter) -> array(varchar)
```
Splits `string` on `delimiter` and returns an array.
```
split(string, delimiter, limit) -> array(varchar)
```
Splits `string` on `delimiter` and returns an array of size at most
`limit`. The last element in the array always contain everything
left in the `string`. `limit` must be a positive number.
### split\_part
```
split_part(string, delimiter, index) -> varchar
```
Splits `string` on `delimiter` and returns the field `index`.
Field indexes start with `1`. If the index is larger than
the number of fields, then null is returned.
### split\_to\_map
```
split_to_map(string, entryDelimiter, keyValueDelimiter) -> map
```
Splits `string` by `entryDelimiter` and `keyValueDelimiter` and returns a map.
`entryDelimiter` splits `string` into key-value pairs. `keyValueDelimiter` splits
each pair into key and value.
### split\_to\_multimap
```
split_to_multimap(string, entryDelimiter, keyValueDelimiter) -> map(varchar, array(varchar))
```
Splits `string` by `entryDelimiter` and `keyValueDelimiter` and returns a map
containing an array of values for each unique key. `entryDelimiter` splits `string`
into key-value pairs. `keyValueDelimiter` splits each pair into key and value. The
values for each key will be in the same order as they appeared in `string`.
### strpos
```
strpos(string, substring) -> bigint
```
Returns the starting position of the first instance of `substring` in
`string`. Positions start with `1`. If not found, `0` is returned.
```
strpos(string, substring, instance) -> bigint
```
Returns the position of the N-th `instance` of `substring` in `string`.
When `instance` is a negative number the search will start from the end of `string`.
Positions start with `1`. If not found, `0` is returned.
### starts\_with
```
starts_with(string, substring) -> boolean
```
Tests whether `substring` is a prefix of `string`.
### substr
```
substr(string, start) -> varchar
```
This is an alias for [substring()](#substring).
```
substr(string, start, length) -> varchar
```
This is an alias for [substring()](#substring).
### substring
```
substring(string, start) -> varchar
```
Returns the rest of `string` from the starting position `start`.
Positions start with `1`. A negative starting position is interpreted
as being relative to the end of the string.
```
substring(string, start, length) -> varchar
```
Returns a substring from `string` of length `length` from the starting
position `start`. Positions start with `1`. A negative starting
position is interpreted as being relative to the end of the string.
### translate
```
translate(source, from, to) -> varchar
```
Returns the `source` string translated by replacing characters found in the
`from` string with the corresponding characters in the `to` string. If the `from`
string contains duplicates, only the first is used. If the `source` character
does not exist in the `from` string, the `source` character will be copied
without translation. If the index of the matching character in the `from`
string is beyond the length of the `to` string, the `source` character will
be omitted from the resulting string.
Here are some examples illustrating the translate function:
```sql theme={null}
SELECT translate('abcd', '', ''); -- 'abcd'
SELECT translate('abcd', 'a', 'z'); -- 'zbcd'
SELECT translate('abcda', 'a', 'z'); -- 'zbcdz'
SELECT translate('Palhoça', 'ç','c'); -- 'Palhoca'
SELECT translate('abcd', 'b', U&'\+01F600'); -- a😀cd
SELECT translate('abcd', 'a', ''); -- 'bcd'
SELECT translate('abcd', 'a', 'zy'); -- 'zbcd'
SELECT translate('abcd', 'ac', 'z'); -- 'zbd'
SELECT translate('abcd', 'aac', 'zq'); -- 'zbd'
```
### trime
```
trim(string) -> varchar
```
Removes leading and trailing whitespace from `string`.
```
trim( [ [ specification ] [ string ] FROM ] source ) -> varchar
```
Removes any leading and/or trailing characters as specified up to and
including `string` from `source`:
```sql theme={null}
SELECT trim('!' FROM '!foo!'); -- 'foo'
SELECT trim(LEADING FROM ' abcd'); -- 'abcd'
SELECT trim(BOTH '$' FROM '$var$'); -- 'var'
SELECT trim(TRAILING 'ER' FROM upper('worker')); -- 'WORK'
```
### upper
```
upper(string) -> varchar
```
Converts `string` to uppercase.
### word\_stem
```
word_stem(word) -> varchar
```
Returns the stem of `word` in the English language.
```
word_stem(word, lang) -> varchar
```
Returns the stem of `word` in the `lang` language.
## Unicode functions
### normalize
```
normalize(string) -> varchar
Transforms `string` with NFC normalization form.
```
```
normalize(string, form) -> varchar
```
Transforms `string` with the specified normalization form.
`form` must be one of the following keywords:
| Form | Description |
| ------ | -------------------------------------------------------------- |
| `NFD` | Canonical Decomposition |
| `NFC` | Canonical Decomposition, followed by Canonical Composition |
| `NFKD` | Compatibility Decomposition |
| `NFKC` | Compatibility Decomposition, followed by Canonical Composition |
This SQL-standard function has special syntax and requires
specifying `form` as a keyword, not as a string.
### to\_utf8
```
to_utf8(string) -> varbinary
```
Encodes `string` into a UTF-8 varbinary representation.
### from\_utf8
```
from_utf8(binary) -> varchar
```
Decodes a UTF-8 encoded string from `binary`. Invalid UTF-8 sequences
are replaced with the Unicode replacement character `U+FFFD`.
```
from_utf8(binary, replace) -> varchar
```
Decodes a UTF-8 encoded string from `binary`. Invalid UTF-8 sequences
are replaced with `replace`. The replacement string `replace` must either
be a single character or empty (in which case invalid characters are
removed).
# Table functions
Source: https://docs.upsolve.ai/sql/functions/table
A table function is a function returning a table. It can be invoked inside the
`FROM` clause of a query:
```sql theme={null}
SELECT * FROM TABLE(my_function(1, 100))
```
The row type of the returned table can depend on the arguments passed with
invocation of the function. If different row types can be returned, the
function is a **polymorphic table function**.
Polymorphic table functions allow you to dynamically invoke custom logic from
within the SQL query. They can be used for working with external systems as
well as for enhancing Trino with capabilities going beyond the SQL standard.
## Built-in table functions
### exclude\_columns
```
exclude_columns(input => table, columns => descriptor) -> table
```
Excludes from `table` all columns listed in `descriptor`:
```sql theme={null}
SELECT *
FROM TABLE(exclude_columns(
input => TABLE(orders),
columns => DESCRIPTOR(clerk, comment)))
```
The argument `input` is a table or a query.
The argument `columns` is a descriptor without types.
```
sequence(start => bigint, stop => bigint, step => bigint) -> table(sequential_number bigint)
```
Returns a single column `sequential_number` containing a sequence of
bigint:
```sql theme={null}
SELECT *
FROM TABLE(sequence(
start => 1000000,
stop => -2000000,
step => -3))
```
`start` is the first element in the sequence. The default value is `0`.
`stop` is the end of the range, inclusive. The last element in the
sequence is equal to `stop`, or it is the last value within range,
reachable by steps.
`step` is the difference between subsequent values. The default value is
`1`.
The result of the `sequence` table function might not be ordered.
## Table function invocation
You invoke a table function in the `FROM` clause of a query. Table function
invocation syntax is similar to a scalar function call.
### Function resolution
Every table function is provided by a catalog, and it belongs to a schema in
the catalog. You can qualify the function name with a schema name, or with
catalog and schema names:
```sql theme={null}
SELECT * FROM TABLE(schema_name.my_function(1, 100))
SELECT * FROM TABLE(catalog_name.schema_name.my_function(1, 100))
```
Otherwise, the standard Trino name resolution is applied. The connection
between the function and the catalog must be identified, because the function
is executed by the corresponding connector. If the function is not registered
by the specified catalog, the query fails.
The table function name is resolved case-insensitive, analogically to scalar
function and table resolution in Trino.
### Arguments
There are three types of arguments.
1. Scalar arguments
They must be constant expressions, and they can be of any SQL type, which is
compatible with the declared argument type:
```
factor => 42
```
2. Descriptor arguments
Descriptors consist of fields with names and optional data types:
```
schema => DESCRIPTOR(id BIGINT, name VARCHAR)
columns => DESCRIPTOR(date, status, comment)
```
To pass `null` for a descriptor, use:
```
schema => CAST(null AS DESCRIPTOR)
```
3. Table arguments
You can pass a table name, or a query. Use the keyword `TABLE`:
```
input => TABLE(orders)
data => TABLE(SELECT * FROM region, nation WHERE region.regionkey = nation.regionkey)
```
If the table argument is declared as set semantics,
you can specify partitioning and ordering. Each partition is processed
independently by the table function. If you do not specify partitioning, the
argument is processed as a single partition. You can also specify
`PRUNE WHEN EMPTY` or `KEEP WHEN EMPTY`. With `PRUNE WHEN EMPTY` you
declare that you are not interested in the function result if the argument is
empty. This information is used by the Trino engine to optimize the query. The
`KEEP WHEN EMPTY` option indicates that the function should be executed even
if the table argument is empty. By specifying `KEEP WHEN EMPTY` or
`PRUNE WHEN EMPTY`, you override the property set for the argument by the
function author.
The following example shows how the table argument properties should be ordered:
```sql theme={null}
input => TABLE(orders)
PARTITION BY orderstatus
KEEP WHEN EMPTY
ORDER BY orderdate
```
### Argument passing conventions
There are two conventions of passing arguments to a table function:
* **Arguments passed by name**:
```sql theme={null}
SELECT * FROM TABLE(my_function(row_count => 100, column_count => 1))
```
In this convention, you can pass the arguments in arbitrary order. Arguments
declared with default values can be skipped. Argument names are resolved
case-sensitive, and with automatic uppercasing of unquoted names.
* **Arguments passed positionally**:
```sql theme={null}
SELECT * FROM TABLE(my_function(1, 100))
```
In this convention, you must follow the order in which the arguments are
declared. You can skip a suffix of the argument list, provided that all the
skipped arguments are declared with default values.
You cannot mix the argument conventions in one invocation.
You can also use parameters in arguments:
```sql theme={null}
PREPARE stmt FROM
SELECT * FROM TABLE(my_function(row_count => ? + 1, column_count => ?));
EXECUTE stmt USING 100, 1;
```
# Teradata functions
Source: https://docs.upsolve.ai/sql/functions/teradata
These functions provide compatibility with Teradata SQL.
## String functions
### char2hexint
```
char2hexint(string) -> varchar
```
Returns the hexadecimal representation of the UTF-16BE encoding of the string.
### index
```
index(string, substring) -> bigint
```
Alias for [strpos()](/sql/functions/string#strpos) function.
## Date functions
The functions in this section use a format string that is compatible with
the Teradata datetime functions. The following table, based on the
Teradata reference manual, describes the supported format specifiers:
| Specifier | Description |
| ------------- | ---------------------------------- |
| `- / , . ; :` | Punctuation characters are ignored |
| `dd` | Day of month (1-31) |
| `hh` | Hour of day (1-12) |
| `hh24` | Hour of the day (0-23) |
| `mi` | Minute (0-59) |
| `mm` | Month (01-12) |
| `ss` | Second (0-59) |
| `yyyy` | 4-digit year |
| `yy` | 2-digit year |
Case insensitivity is not currently supported. All specifiers must be lowercase.
### to\_char
```
to_char(timestamp, format) -> varchar
```
Formats `timestamp` as a string using `format`.
### to\_timestamp
```
to_timestamp(string, format) -> timestamp
```
Parses `string` into a `TIMESTAMP` using `format`.
### to\_date
```
to_date(string, format) -> date
```
Parses `string` into a `DATE` using `format`.
# URL functions
Source: https://docs.upsolve.ai/sql/functions/url
## Extraction functions
The URL extraction functions extract components from HTTP URLs
(or any valid URIs conforming to RFC 2396).
The following syntax is supported:
```
[protocol:][//host[:port]][path][?query][#fragment]
```
The extracted components do not contain URI syntax separators
such as `:` or `?`.
### url\_extract\_fragment
```
url_extract_fragment(url) -> varchar
```
Returns the fragment identifier from `url`.
### url\_extract\_host
```
url_extract_host(url) -> varchar
```
Returns the host from `url`.
### url\_extract\_parameter
```
url_extract_parameter(url, name) -> varchar
```
Returns the value of the first query string parameter named `name`
from `url`. Parameter extraction is handled in the typical manner
as specified by RFC 1866.
### url\_extract\_path
```
url_extract_path(url) -> varchar
```
Returns the path from `url`.
### url\_extract\_port
```
url_extract_port(url) -> bigint
```
Returns the port number from `url`.
### url\_extract\_protocol
```
url_extract_protocol(url) -> varchar
```
Returns the protocol from `url`:
```sql theme={null}
SELECT url_extract_protocol('http://localhost:8080/req_path');
-- http
SELECT url_extract_protocol('https://127.0.0.1:8080/req_path');
-- https
SELECT url_extract_protocol('ftp://path/file');
-- ftp
```
### url\_extract\_query
```
url_extract_query(url) -> varchar
```
Returns the query string from `url`.
## Encoding functions
### url\_encode
```
url_encode(value) -> varchar
```
Escapes `value` by encoding it so that it can be safely included in
URL query parameter names and values:
* Alphanumeric characters are not encoded.
* The characters `.`, `-`, `*` and `_` are not encoded.
* The ASCII space character is encoded as `+`.
* All other characters are converted to UTF-8 and the bytes are encoded
as the string `%XX` where `XX` is the uppercase hexadecimal
value of the UTF-8 byte.
### url\_decode
```
url_decode(value) -> varchar
```
Unescapes the URL encoded `value`.
This function is the inverse of [url\_encode()](#url-encode).
# UUID Functions
Source: https://docs.upsolve.ai/sql/functions/uuid
### uuid
```
uuid() -> uuid
```
Returns a pseudo randomly generated [UUID](/sql/reference/types#uuid-type) (type 4).
# Window functions
Source: https://docs.upsolve.ai/sql/functions/window
Window functions perform calculations across rows of the query result.
They run after the `HAVING` clause but before the `ORDER BY` clause.
Invoking a window function requires special syntax using the `OVER`
clause to specify the window.
For example, the following query ranks orders for each clerk by price:
```sql theme={null}
SELECT orderkey, clerk, totalprice,
rank() OVER (PARTITION BY clerk
ORDER BY totalprice DESC) AS rnk
FROM orders
ORDER BY clerk, rnk
```
The window can be specified in two ways (see [WINDOW clause](/sql/reference/select#window-clause)):
* By a reference to a named window specification defined in the `WINDOW` clause,
* By an in-line window specification which allows to define window components
as well as refer to the window components pre-defined in the `WINDOW` clause.
## Aggregate functions
All [aggregate functions](/sql/functions/aggregate) can be used as window functions by adding the `OVER`
clause. The aggregate function is computed for each row over the rows within the
current row's window frame. Note that [ordering during
aggregation](/sql/functions/aggregate#ordering-during-aggregation) is not supported.
For example, the following query produces a rolling sum of order prices
by day for each clerk:
```sql theme={null}
SELECT clerk, orderdate, orderkey, totalprice,
sum(totalprice) OVER (PARTITION BY clerk
ORDER BY orderdate) AS rolling_sum
FROM orders
ORDER BY clerk, orderdate, orderkey
```
## Ranking functions
### cume\_dist
```
cume_dist() -> bigint
```
Returns the cumulative distribution of a value in a group of values.
The result is the number of rows preceding or peer with the row in the
window ordering of the window partition divided by the total number of
rows in the window partition. Thus, any tie values in the ordering will
evaluate to the same distribution value.
### dense\_rank
```
dense_rank() -> bigint
```
Returns the rank of a value in a group of values. This is similar to
[rank()](#rank), except that tie values do not produce gaps in the sequence.
### ntile
```
ntile(n) -> bigint
```
Divides the rows for each window partition into `n` buckets ranging
from `1` to at most `n`. Bucket values will differ by at most `1`.
If the number of rows in the partition does not divide evenly into the
number of buckets, then the remainder values are distributed one per
bucket, starting with the first bucket.
For example, with `6` rows and `4` buckets, the bucket values would
be as follows: `1` `1` `2` `2` `3` `4`
### percent\_rank
```
percent_rank() -> double
```
Returns the percentage ranking of a value in group of values. The result
is `(r - 1) / (n - 1)` where `r` is the [rank()](#rank) of the row and
`n` is the total number of rows in the window partition.
### rank
```
rank() -> bigint
```
Returns the rank of a value in a group of values. The rank is one plus
the number of rows preceding the row that are not peer with the row.
Thus, tie values in the ordering will produce gaps in the sequence.
The ranking is performed for each window partition.
### row\_number
```
row_number() -> bigint
```
Returns a unique, sequential number for each row, starting with one,
according to the ordering of rows within the window partition.
## Value functions
By default, null values are respected. If `IGNORE NULLS` is specified, all rows where
`x` is null are excluded from the calculation. If `IGNORE NULLS` is specified and `x`
is null for all rows, the `default_value` is returned, or if it is not specified,
`null` is returned.
### first\_value
```
first_value(x) -> [same as input]
```
Returns the first value of the window.
### last\_value
```
last_value(x) -> [same as input]
```
Returns the last value of the window.
### nth\_value
```
nth_value(x, offset) -> [same as input]
```
Returns the value at the specified offset from the beginning of the window.
Offsets start at `1`. The offset can be any scalar
expression. If the offset is null or greater than the number of values in
the window, `null` is returned. It is an error for the offset to be zero or
negative.
### lead
```
lead(x[, offset [, default_value]]) -> [same as input]
```
Returns the value at `offset` rows after the current row in the window partition.
Offsets start at `0`, which is the current row. The
offset can be any scalar expression. The default `offset` is `1`. If the
offset is null, an error is raised. If the offset refers to a row that is not
within the partition, the `default_value` is returned, or if it is not specified
`null` is returned.
The `lead` function requires that the window ordering be specified.
Window frame must not be specified.
### lag
```
lag(x[, offset [, default_value]]) -> [same as input]
```
Returns the value at `offset` rows before the current row in the window partition.
Offsets start at `0`, which is the current row. The
offset can be any scalar expression. The default `offset` is `1`. If the
offset is null, an error is raised. If the offset refers to a row that is not
within the partition, the `default_value` is returned, or if it is not specified
`null` is returned.
The `lag` function requires that the window ordering be specified.
Window frame must not be specified.
# Introduction
Source: https://docs.upsolve.ai/sql/introduction
Introduction to how to use SQL with Upsolve.
## Welcome
Upsolve uses Trino's ANSI SQL compliant query engine. This standard compliance allows Upsolve users to integrate their
favorite data tools, including BI and ETL tools with any underlying data source.
Our query engine validates and translates the received SQL statements into the necessary operations on the connected
data source.
This section provides a reference to the supported SQL data types and other general characteristics of the SQL support
of Upsolve.
# DESCRIBE
Source: https://docs.upsolve.ai/sql/reference/describe
## Synopsis
```sql theme={null}
DESCRIBE table_name
```
## Description
`DESCRIBE` is an alias for [SHOW COLUMNS](/sql/reference/show-columns).
# EXECUTE
Source: https://docs.upsolve.ai/sql/reference/execute
## Synopsis
```sql theme={null}
EXECUTE statement_name [ USING parameter1 [ , parameter2, ... ] ]
```
## Description
Executes a prepared statement with the name `statement_name`. Parameter values
are defined in the `USING` clause.
## Examples
Prepare and execute a query with no parameters:
```sql theme={null}
PREPARE my_select1 FROM
SELECT name FROM nation;
```
```sql theme={null}
EXECUTE my_select1;
```
Prepare and execute a query with two parameters:
```sql theme={null}
PREPARE my_select2 FROM
SELECT name FROM nation WHERE regionkey = ? and nationkey < ?;
```
```sql theme={null}
EXECUTE my_select2 USING 1, 3;
```
This is equivalent to:
```sql theme={null}
SELECT name FROM nation WHERE regionkey = 1 AND nationkey < 3;
```
## See also
[PREPARE](/sql/reference/prepare)
# PREPARE
Source: https://docs.upsolve.ai/sql/reference/prepare
## Synopsis
```text theme={null}
PREPARE statement_name FROM statement
```
## Description
Prepares a statement for execution at a later time. Prepared statements are
queries that are saved in a session with a given name. The statement can
include parameters in place of literals to be replaced at execution time.
Parameters are represented by question marks.
## Examples
Prepare a select query:
```sql theme={null}
PREPARE my_select1 FROM
SELECT * FROM nation;
```
Prepare a select query that includes parameters. The values to compare with
`regionkey` and `nationkey` will be filled in with the [EXECUTE](/sql/reference/execute) statement:
```sql theme={null}
PREPARE my_select2 FROM
SELECT name FROM nation WHERE regionkey = ? AND nationkey < ?;
```
Prepare an insert query:
```sql theme={null}
PREPARE my_insert FROM
INSERT INTO cities VALUES (1, 'San Francisco');
```
## See also
[EXECUTE](/sql/reference/execute)
# SELECT
Source: https://docs.upsolve.ai/sql/reference/select
## Synopsis
```sql theme={null}
[ WITH FUNCTION sql_routines ]
[ WITH [ RECURSIVE ] with_query [, ...] ]
SELECT [ ALL | DISTINCT ] select_expression [, ...]
[ FROM from_item [, ...] ]
[ WHERE condition ]
[ GROUP BY [ ALL | DISTINCT ] grouping_element [, ...] ]
[ HAVING condition]
[ WINDOW window_definition_list]
[ { UNION | INTERSECT | EXCEPT } [ ALL | DISTINCT ] select ]
[ ORDER BY expression [ ASC | DESC ] [, ...] ]
[ OFFSET count [ ROW | ROWS ] ]
[ LIMIT { count | ALL } ]
[ FETCH { FIRST | NEXT } [ count ] { ROW | ROWS } { ONLY | WITH TIES } ]
```
where `from_item` is one of
```sql theme={null}
table_name [ [ AS ] alias [ ( column_alias [, ...] ) ] ]
```
```sql theme={null}
from_item join_type from_item
[ ON join_condition | USING ( join_column [, ...] ) ]
```
```sql theme={null}
TABLE (table_function_invocation) [ [ AS ] alias [ ( column_alias [, ...] ) ] ]
```
For description of table functions usage, see [table functions](/sql/functions/table).
and `join_type` is one of
```sql theme={null}
[ INNER ] JOIN
LEFT [ OUTER ] JOIN
RIGHT [ OUTER ] JOIN
FULL [ OUTER ] JOIN
CROSS JOIN
```
and `grouping_element` is one of
```sql theme={null}
()
expression
GROUPING SETS ( ( column [, ...] ) [, ...] )
CUBE ( column [, ...] )
ROLLUP ( column [, ...] )
```
## Description
Retrieve rows from zero or more tables.
## WITH FUNCTION clause
The `WITH FUNCTION` clause allows you to define a list of inline SQL routines
that are available for use in the rest of the query.
The following example declares and uses two inline routines:
```sql theme={null}
WITH
FUNCTION hello(name varchar)
RETURNS varchar
RETURN format('Hello %s!', 'name'),
FUNCTION bye(name varchar)
RETURNS varchar
RETURN format('Bye %s!', 'name')
SELECT hello('Finn') || ' and ' || bye('Joe');
-- Hello Finn! and Bye Joe!
```
Find further information about routines in general, inline routines, all
supported statements, and examples in [routines](/sql/sql/routines/introduction).
## WITH clause
The `WITH` clause defines named relations for use within a query.
It allows flattening nested queries or simplifying subqueries.
For example, the following queries are equivalent:
```sql theme={null}
SELECT a, b
FROM (
SELECT a, MAX(b) AS b FROM t GROUP BY a
) AS x;
WITH x AS (SELECT a, MAX(b) AS b FROM t GROUP BY a)
SELECT a, b FROM x;
```
This also works with multiple subqueries:
```sql theme={null}
WITH
t1 AS (SELECT a, MAX(b) AS b FROM x GROUP BY a),
t2 AS (SELECT a, AVG(d) AS d FROM y GROUP BY a)
SELECT t1.*, t2.*
FROM t1
JOIN t2 ON t1.a = t2.a;
```
Additionally, the relations within a `WITH` clause can chain:
```sql theme={null}
WITH
x AS (SELECT a FROM t),
y AS (SELECT a AS b FROM x),
z AS (SELECT b AS c FROM y)
SELECT c FROM z;
```
Currently, the SQL for the `WITH` clause will be inlined anywhere the named
relation is used. This means that if the relation is used more than once and the query
is non-deterministic, the results may be different each time.
## WITH RECURSIVE clause
The `WITH RECURSIVE` clause is a variant of the `WITH` clause. It defines
a list of queries to process, including recursive processing of suitable
queries.
This feature is experimental only. Proceed to use it only if you understand
potential query failures and the impact of the recursion processing on your
workload.
A recursive `WITH`-query must be shaped as a `UNION` of two relations. The
first relation is called the *recursion base*, and the second relation is called
the *recursion step*. Trino supports recursive `WITH`-queries with a single
recursive reference to a `WITH`-query from within the query. The name `T` of
the query `T` can be mentioned once in the `FROM` clause of the recursion
step relation.
The following listing shows a simple example, that displays a commonly used
form of a single query in the list:
```sql theme={null}
WITH RECURSIVE t(n) AS (
VALUES (1)
UNION ALL
SELECT n + 1 FROM t WHERE n < 4
)
SELECT sum(n) FROM t;
```
In the preceding query the simple assignment `VALUES (1)` defines the
recursion base relation. `SELECT n + 1 FROM t WHERE n < 4` defines the
recursion step relation. The recursion processing performs these steps:
* recursive base yields `1`
* first recursion yields `1 + 1 = 2`
* second recursion uses the result from the first and adds one: `2 + 1 = 3`
* third recursion uses the result from the second and adds one again:
`3 + 1 = 4`
* fourth recursion aborts since `n = 4`
* this results in `t` having values `1`, `2`, `3` and `4`
* the final statement performs the sum operation of these elements with the
final result value `10`
The types of the returned columns are those of the base relation. Therefore it
is required that types in the step relation can be coerced to base relation
types.
The `RECURSIVE` clause applies to all queries in the `WITH` list, but not
all of them must be recursive. If a `WITH`-query is not shaped according to
the rules mentioned above or it does not contain a recursive reference, it is
processed like a regular `WITH`-query. Column aliases are mandatory for all
the queries in the recursive `WITH` list.
The following limitations apply as a result of following the SQL standard and
due to implementation choices, in addition to `WITH` clause limitations:
* only single-element recursive cycles are supported. Like in regular
`WITH`-queries, references to previous queries in the `WITH` list are
allowed. References to following queries are forbidden.
* usage of outer joins, set operations, limit clause, and others is not always
allowed in the step relation
* recursion depth is fixed, defaults to `10`, and doesn't depend on the actual
query results
## SELECT clause
The `SELECT` clause specifies the output of the query. Each `select_expression`
defines a column or columns to be included in the result.
```sql theme={null}
SELECT [ ALL | DISTINCT ] select_expression [, ...]
```
The `ALL` and `DISTINCT` quantifiers determine whether duplicate rows
are included in the result set. If the argument `ALL` is specified,
all rows are included. If the argument `DISTINCT` is specified, only unique
rows are included in the result set. In this case, each output column must
be of a type that allows comparison. If neither argument is specified,
the behavior defaults to `ALL`.
### Select expressions
Each `select_expression` must be in one of the following forms:
```sql theme={null}
expression [ [ AS ] column_alias ]
```
```sql theme={null}
row_expression.* [ AS ( column_alias [, ...] ) ]
```
```sql theme={null}
relation.*
```
```sql theme={null}
*
```
In the case of `expression [ [ AS ] column_alias ]`, a single output column
is defined.
In the case of `row_expression.* [ AS ( column_alias [, ...] ) ]`,
the `row_expression` is an arbitrary expression of type `ROW`.
All fields of the row define output columns to be included in the result set.
In the case of `relation.*`, all columns of `relation` are included
in the result set. In this case column aliases are not allowed.
In the case of `*`, all columns of the relation defined by the query
are included in the result set.
In the result set, the order of columns is the same as the order of their
specification by the select expressions. If a select expression returns multiple
columns, they are ordered the same way they were ordered in the source
relation or row type expression.
If column aliases are specified, they override any preexisting column
or row field names:
```sql theme={null}
SELECT (CAST(ROW(1, true) AS ROW(field1 bigint, field2 boolean))).* AS (alias1, alias2);
```
```sql theme={null}
alias1 | alias2
--------+--------
1 | true
(1 row)
```
Otherwise, the existing names are used:
```sql theme={null}
SELECT (CAST(ROW(1, true) AS ROW(field1 bigint, field2 boolean))).*;
```
```sql theme={null}
field1 | field2
--------+--------
1 | true
(1 row)
```
and in their absence, anonymous columns are produced:
```sql theme={null}
SELECT (ROW(1, true)).*;
```
```sql theme={null}
_col0 | _col1
-------+-------
1 | true
(1 row)
```
## GROUP BY clause
The `GROUP BY` clause divides the output of a `SELECT` statement into
groups of rows containing matching values. A simple `GROUP BY` clause may
contain any expression composed of input columns or it may be an ordinal
number selecting an output column by position (starting at one).
The following queries are equivalent. They both group the output by
the `nationkey` input column with the first query using the ordinal
position of the output column and the second query using the input
column name:
```sql theme={null}
SELECT count(*), nationkey FROM customer GROUP BY 2;
SELECT count(*), nationkey FROM customer GROUP BY nationkey;
```
`GROUP BY` clauses can group output by input column names not appearing in
the output of a select statement. For example, the following query generates
row counts for the `customer` table using the input column `mktsegment`:
```sql theme={null}
SELECT count(*) FROM customer GROUP BY mktsegment;
```
```sql theme={null}
_col0
-------
29968
30142
30189
29949
29752
(5 rows)
```
When a `GROUP BY` clause is used in a `SELECT` statement all output
expressions must be either aggregate functions or columns present in
the `GROUP BY` clause.
### Complex grouping operations
Trino also supports complex aggregations using the `GROUPING SETS`, `CUBE`
and `ROLLUP` syntax. This syntax allows users to perform analysis that requires
aggregation on multiple sets of columns in a single query. Complex grouping
operations do not support grouping on expressions composed of input columns.
Only column names are allowed.
Complex grouping operations are often equivalent to a `UNION ALL` of simple
`GROUP BY` expressions, as shown in the following examples. This equivalence
does not apply, however, when the source of data for the aggregation
is non-deterministic.
### GROUPING SETS
Grouping sets allow users to specify multiple lists of columns to group on.
The columns not part of a given sublist of grouping columns are set to `NULL`.
```sql theme={null}
SELECT * FROM shipping;
```
```sql theme={null}
origin_state | origin_zip | destination_state | destination_zip | package_weight
--------------+------------+-------------------+-----------------+----------------
California | 94131 | New Jersey | 8648 | 13
California | 94131 | New Jersey | 8540 | 42
New Jersey | 7081 | Connecticut | 6708 | 225
California | 90210 | Connecticut | 6927 | 1337
California | 94131 | Colorado | 80302 | 5
New York | 10002 | New Jersey | 8540 | 3
(6 rows)
```
`GROUPING SETS` semantics are demonstrated by this example query:
```sql theme={null}
SELECT origin_state, origin_zip, destination_state, sum(package_weight)
FROM shipping
GROUP BY GROUPING SETS (
(origin_state),
(origin_state, origin_zip),
(destination_state));
```
```sql theme={null}
origin_state | origin_zip | destination_state | _col0
--------------+------------+-------------------+-------
New Jersey | NULL | NULL | 225
California | NULL | NULL | 1397
New York | NULL | NULL | 3
California | 90210 | NULL | 1337
California | 94131 | NULL | 60
New Jersey | 7081 | NULL | 225
New York | 10002 | NULL | 3
NULL | NULL | Colorado | 5
NULL | NULL | New Jersey | 58
NULL | NULL | Connecticut | 1562
(10 rows)
```
The preceding query may be considered logically equivalent to a `UNION ALL` of
multiple `GROUP BY` queries:
```sql theme={null}
SELECT origin_state, NULL, NULL, sum(package_weight)
FROM shipping GROUP BY origin_state
UNION ALL
SELECT origin_state, origin_zip, NULL, sum(package_weight)
FROM shipping GROUP BY origin_state, origin_zip
UNION ALL
SELECT NULL, NULL, destination_state, sum(package_weight)
FROM shipping GROUP BY destination_state;
```
However, the query with the complex grouping syntax (`GROUPING SETS`, `CUBE`
or `ROLLUP`) will only read from the underlying data source once, while the
query with the `UNION ALL` reads the underlying data three times. This is why
queries with a `UNION ALL` may produce inconsistent results when the data
source is not deterministic.
### CUBE
The `CUBE` operator generates all possible grouping sets (i.e. a power set)
for a given set of columns. For example, the query:
```sql theme={null}
SELECT origin_state, destination_state, sum(package_weight)
FROM shipping
GROUP BY CUBE (origin_state, destination_state);
```
is equivalent to:
```sql theme={null}
SELECT origin_state, destination_state, sum(package_weight)
FROM shipping
GROUP BY GROUPING SETS (
(origin_state, destination_state),
(origin_state),
(destination_state),
()
);
```
```sql theme={null}
origin_state | destination_state | _col0
--------------+-------------------+-------
California | New Jersey | 55
California | Colorado | 5
New York | New Jersey | 3
New Jersey | Connecticut | 225
California | Connecticut | 1337
California | NULL | 1397
New York | NULL | 3
New Jersey | NULL | 225
NULL | New Jersey | 58
NULL | Connecticut | 1562
NULL | Colorado | 5
NULL | NULL | 1625
(12 rows)
```
### ROLLUP
The `ROLLUP` operator generates all possible subtotals for a given set of
columns. For example, the query:
```sql theme={null}
SELECT origin_state, origin_zip, sum(package_weight)
FROM shipping
GROUP BY ROLLUP (origin_state, origin_zip);
```
```sql theme={null}
origin_state | origin_zip | _col2
--------------+------------+-------
California | 94131 | 60
California | 90210 | 1337
New Jersey | 7081 | 225
New York | 10002 | 3
California | NULL | 1397
New York | NULL | 3
New Jersey | NULL | 225
NULL | NULL | 1625
(8 rows)
```
is equivalent to:
```sql theme={null}
SELECT origin_state, origin_zip, sum(package_weight)
FROM shipping
GROUP BY GROUPING SETS ((origin_state, origin_zip), (origin_state), ());
```
### Combining multiple grouping expressions
Multiple grouping expressions in the same query are interpreted as having
cross-product semantics. For example, the following query:
```sql theme={null}
SELECT origin_state, destination_state, origin_zip, sum(package_weight)
FROM shipping
GROUP BY
GROUPING SETS ((origin_state, destination_state)),
ROLLUP (origin_zip);
```
which can be rewritten as:
```sql theme={null}
SELECT origin_state, destination_state, origin_zip, sum(package_weight)
FROM shipping
GROUP BY
GROUPING SETS ((origin_state, destination_state)),
GROUPING SETS ((origin_zip), ());
```
is logically equivalent to:
```sql theme={null}
SELECT origin_state, destination_state, origin_zip, sum(package_weight)
FROM shipping
GROUP BY GROUPING SETS (
(origin_state, destination_state, origin_zip),
(origin_state, destination_state)
);
```
```sql theme={null}
origin_state | destination_state | origin_zip | _col3
--------------+-------------------+------------+-------
New York | New Jersey | 10002 | 3
California | New Jersey | 94131 | 55
New Jersey | Connecticut | 7081 | 225
California | Connecticut | 90210 | 1337
California | Colorado | 94131 | 5
New York | New Jersey | NULL | 3
New Jersey | Connecticut | NULL | 225
California | Colorado | NULL | 5
California | Connecticut | NULL | 1337
California | New Jersey | NULL | 55
(10 rows)
```
The `ALL` and `DISTINCT` quantifiers determine whether duplicate grouping
sets each produce distinct output rows. This is particularly useful when
multiple complex grouping sets are combined in the same query. For example, the
following query:
```sql theme={null}
SELECT origin_state, destination_state, origin_zip, sum(package_weight)
FROM shipping
GROUP BY ALL
CUBE (origin_state, destination_state),
ROLLUP (origin_state, origin_zip);
```
is equivalent to:
```sql theme={null}
SELECT origin_state, destination_state, origin_zip, sum(package_weight)
FROM shipping
GROUP BY GROUPING SETS (
(origin_state, destination_state, origin_zip),
(origin_state, origin_zip),
(origin_state, destination_state, origin_zip),
(origin_state, origin_zip),
(origin_state, destination_state),
(origin_state),
(origin_state, destination_state),
(origin_state),
(origin_state, destination_state),
(origin_state),
(destination_state),
()
);
```
However, if the query uses the `DISTINCT` quantifier for the `GROUP BY`:
```sql theme={null}
SELECT origin_state, destination_state, origin_zip, sum(package_weight)
FROM shipping
GROUP BY DISTINCT
CUBE (origin_state, destination_state),
ROLLUP (origin_state, origin_zip);
```
only unique grouping sets are generated:
```sql theme={null}
SELECT origin_state, destination_state, origin_zip, sum(package_weight)
FROM shipping
GROUP BY GROUPING SETS (
(origin_state, destination_state, origin_zip),
(origin_state, origin_zip),
(origin_state, destination_state),
(origin_state),
(destination_state),
()
);
```
The default set quantifier is `ALL`.
### GROUPING operation
`grouping(col1, ..., colN) -> bigint`
The grouping operation returns a bit set converted to decimal, indicating which columns are present in a
grouping. It must be used in conjunction with `GROUPING SETS`, `ROLLUP`, `CUBE` or `GROUP BY`
and its arguments must match exactly the columns referenced in the corresponding `GROUPING SETS`,
`ROLLUP`, `CUBE` or `GROUP BY` clause.
To compute the resulting bit set for a particular row, bits are assigned to the argument columns with
the rightmost column being the least significant bit. For a given grouping, a bit is set to 0 if the
corresponding column is included in the grouping and to 1 otherwise. For example, consider the query
below:
```sql theme={null}
SELECT origin_state, origin_zip, destination_state, sum(package_weight),
grouping(origin_state, origin_zip, destination_state)
FROM shipping
GROUP BY GROUPING SETS (
(origin_state),
(origin_state, origin_zip),
(destination_state)
);
```
```sql theme={null}
origin_state | origin_zip | destination_state | _col3 | _col4
--------------+------------+-------------------+-------+-------
California | NULL | NULL | 1397 | 3
New Jersey | NULL | NULL | 225 | 3
New York | NULL | NULL | 3 | 3
California | 94131 | NULL | 60 | 1
New Jersey | 7081 | NULL | 225 | 1
California | 90210 | NULL | 1337 | 1
New York | 10002 | NULL | 3 | 1
NULL | NULL | New Jersey | 58 | 6
NULL | NULL | Connecticut | 1562 | 6
NULL | NULL | Colorado | 5 | 6
(10 rows)
```
The first grouping in the above result only includes the `origin_state` column and excludes
the `origin_zip` and `destination_state` columns. The bit set constructed for that grouping
is `011` where the most significant bit represents `origin_state`.
## HAVING clause
The `HAVING` clause is used in conjunction with aggregate functions and
the `GROUP BY` clause to control which groups are selected. A `HAVING`
clause eliminates groups that do not satisfy the given conditions.
`HAVING` filters groups after groups and aggregates are computed.
The following example queries the `customer` table and selects groups
with an account balance greater than the specified value:
```sql theme={null}
SELECT count(*), mktsegment, nationkey,
CAST(sum(acctbal) AS bigint) AS totalbal
FROM customer
GROUP BY mktsegment, nationkey
HAVING sum(acctbal) > 5700000
ORDER BY totalbal DESC;
```
```sql theme={null}
_col0 | mktsegment | nationkey | totalbal
-------+------------+-----------+----------
1272 | AUTOMOBILE | 19 | 5856939
1253 | FURNITURE | 14 | 5794887
1248 | FURNITURE | 9 | 5784628
1243 | FURNITURE | 12 | 5757371
1231 | HOUSEHOLD | 3 | 5753216
1251 | MACHINERY | 2 | 5719140
1247 | FURNITURE | 8 | 5701952
(7 rows)
```
(window-clause)=
## WINDOW clause
The `WINDOW` clause is used to define named window specifications. The defined named
window specifications can be referred to in the `SELECT` and `ORDER BY` clauses
of the enclosing query:
```sql theme={null}
SELECT orderkey, clerk, totalprice,
rank() OVER w AS rnk
FROM orders
WINDOW w AS (PARTITION BY clerk ORDER BY totalprice DESC)
ORDER BY count() OVER w, clerk, rnk
```
The window definition list of `WINDOW` clause can contain one or multiple named window
specifications of the form
```sql theme={null}
window_name AS (window_specification)
```
A window specification has the following components:
* The existing window name, which refers to a named window specification in the
`WINDOW` clause. The window specification associated with the referenced name
is the basis of the current specification.
* The partition specification, which separates the input rows into different
partitions. This is analogous to how the `GROUP BY` clause separates rows
into different groups for aggregate functions.
* The ordering specification, which determines the order in which input rows
will be processed by the window function.
* The window frame, which specifies a sliding window of rows to be processed
by the function for a given row. If the frame is not specified, it defaults
to `RANGE UNBOUNDED PRECEDING`, which is the same as
`RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW`. This frame contains all
rows from the start of the partition up to the last peer of the current row.
In the absence of `ORDER BY`, all rows are considered peers, so `RANGE
BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW` is equivalent to `BETWEEN
UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING`. The window frame syntax
supports additional clauses for row pattern recognition. If the row pattern
recognition clauses are specified, the window frame for a particular row
consists of the rows matched by a pattern starting from that row.
Additionally, if the frame specifies row pattern measures, they can be
called over the window, similarly to window functions.
Each window component is optional. If a window specification does not specify
window partitioning, ordering or frame, those components are obtained from
the window specification referenced by the `existing window name`, or from
another window specification in the reference chain. In case when there is no
`existing window name` specified, or none of the referenced window
specifications contains the component, the default value is used.
## Set operations
`UNION` `INTERSECT` and `EXCEPT` are all set operations. These clauses are used
to combine the results of more than one select statement into a single result set:
```sql theme={null}
query UNION [ALL | DISTINCT] query
```
```sql theme={null}
query INTERSECT [ALL | DISTINCT] query
```
```sql theme={null}
query EXCEPT [ALL | DISTINCT] query
```
The argument `ALL` or `DISTINCT` controls which rows are included in
the final result set. If the argument `ALL` is specified all rows are
included even if the rows are identical. If the argument `DISTINCT`
is specified only unique rows are included in the combined result set.
If neither is specified, the behavior defaults to `DISTINCT`.
Multiple set operations are processed left to right, unless the order is explicitly
specified via parentheses. Additionally, `INTERSECT` binds more tightly
than `EXCEPT` and `UNION`. That means `A UNION B INTERSECT C EXCEPT D`
is the same as `A UNION (B INTERSECT C) EXCEPT D`.
### UNION clause
`UNION` combines all the rows that are in the result set from the
first query with those that are in the result set for the second query.
The following is an example of one of the simplest possible `UNION` clauses.
It selects the value `13` and combines this result set with a second query
that selects the value `42`:
```sql theme={null}
SELECT 13
UNION
SELECT 42;
```
```sql theme={null}
_col0
-------
13
42
(2 rows)
```
The following query demonstrates the difference between `UNION` and `UNION ALL`.
It selects the value `13` and combines this result set with a second query that
selects the values `42` and `13`:
```sql theme={null}
SELECT 13
UNION
SELECT * FROM (VALUES 42, 13);
```
```sql theme={null}
_col0
-------
13
42
(2 rows)
```
```sql theme={null}
SELECT 13
UNION ALL
SELECT * FROM (VALUES 42, 13);
```
```sql theme={null}
_col0
-------
13
42
13
(2 rows)
```
### INTERSECT clause
`INTERSECT` returns only the rows that are in the result sets of both the first and
the second queries. The following is an example of one of the simplest
possible `INTERSECT` clauses. It selects the values `13` and `42` and combines
this result set with a second query that selects the value `13`. Since `42`
is only in the result set of the first query, it is not included in the final results.:
```
SELECT * FROM (VALUES 13, 42)
INTERSECT
SELECT 13;
```
```sql theme={null}
_col0
-------
13
(2 rows)
```
### EXCEPT clause
`EXCEPT` returns the rows that are in the result set of the first query,
but not the second. The following is an example of one of the simplest
possible `EXCEPT` clauses. It selects the values `13` and `42` and combines
this result set with a second query that selects the value `13`. Since `13`
is also in the result set of the second query, it is not included in the final result.:
```sql theme={null}
SELECT * FROM (VALUES 13, 42)
EXCEPT
SELECT 13;
```
```sql theme={null}
_col0
-------
42
(2 rows)
```
(order-by-clause)=
## ORDER BY clause
The `ORDER BY` clause is used to sort a result set by one or more
output expressions:
```sql theme={null}
ORDER BY expression [ ASC | DESC ] [ NULLS { FIRST | LAST } ] [, ...]
```
Each expression may be composed of output columns, or it may be an ordinal
number selecting an output column by position, starting at one. The
`ORDER BY` clause is evaluated after any `GROUP BY` or `HAVING` clause,
and before any `OFFSET`, `LIMIT` or `FETCH FIRST` clause.
The default null ordering is `NULLS LAST`, regardless of the ordering direction.
Note that, following the SQL specification, an `ORDER BY` clause only
affects the order of rows for queries that immediately contain the clause.
Trino follows that specification, and drops redundant usage of the clause to
avoid negative performance impacts.
In the following example, the clause only applies to the select statement.
```sql theme={null}
INSERT INTO some_table
SELECT * FROM another_table
ORDER BY field;
```
Since tables in SQL are inherently unordered, and the `ORDER BY` clause in
this case does not result in any difference, but negatively impacts performance
of running the overall insert statement, Trino skips the sort operation.
Another example where the `ORDER BY` clause is redundant, and does not affect
the outcome of the overall statement, is a nested query:
```sql theme={null}
SELECT *
FROM some_table
JOIN (SELECT * FROM another_table ORDER BY field) u
ON some_table.key = u.key;
```
More background information and details can be found in
[a blog post about this optimization](https://trino.io/blog/2019/06/03/redundant-order-by.html).
(offset-clause)=
## OFFSET clause
The `OFFSET` clause is used to discard a number of leading rows
from the result set:
```sql theme={null}
OFFSET count [ ROW | ROWS ]
```
If the `ORDER BY` clause is present, the `OFFSET` clause is evaluated
over a sorted result set, and the set remains sorted after the
leading rows are discarded:
```
SELECT name FROM nation ORDER BY name OFFSET 22;
```
```sql theme={null}
name
----------------
UNITED KINGDOM
UNITED STATES
VIETNAM
(3 rows)
```
Otherwise, it is arbitrary which rows are discarded.
If the count specified in the `OFFSET` clause equals or exceeds the size
of the result set, the final result is empty.
(limit-clause)=
## LIMIT or FETCH FIRST clause
The `LIMIT` or `FETCH FIRST` clause restricts the number of rows
in the result set.
```sql theme={null}
LIMIT { count | ALL }
```
```sql theme={null}
FETCH { FIRST | NEXT } [ count ] { ROW | ROWS } { ONLY | WITH TIES }
```
The following example queries a large table, but the `LIMIT` clause
restricts the output to only have five rows (because the query lacks an `ORDER BY`,
exactly which rows are returned is arbitrary):
```sql theme={null}
SELECT orderdate FROM orders LIMIT 5;
```
```sql theme={null}
orderdate
------------
1994-07-25
1993-11-12
1992-10-06
1994-01-04
1997-12-28
(5 rows)
```
`LIMIT ALL` is the same as omitting the `LIMIT` clause.
The `FETCH FIRST` clause supports either the `FIRST` or `NEXT` keywords
and the `ROW` or `ROWS` keywords. These keywords are equivalent and
the choice of keyword has no effect on query execution.
If the count is not specified in the `FETCH FIRST` clause, it defaults to `1`:
```
SELECT orderdate FROM orders FETCH FIRST ROW ONLY;
```
```sql theme={null}
orderdate
------------
1994-02-12
(1 row)
```
If the `OFFSET` clause is present, the `LIMIT` or `FETCH FIRST` clause
is evaluated after the `OFFSET` clause:
```
SELECT * FROM (VALUES 5, 2, 4, 1, 3) t(x) ORDER BY x OFFSET 2 LIMIT 2;
```
```sql theme={null}
x
---
3
4
(2 rows)
```
For the `FETCH FIRST` clause, the argument `ONLY` or `WITH TIES`
controls which rows are included in the result set.
If the argument `ONLY` is specified, the result set is limited to the exact
number of leading rows determined by the count.
If the argument `WITH TIES` is specified, it is required that the `ORDER BY`
clause be present. The result set consists of the same set of leading rows
and all of the rows in the same peer group as the last of them ('ties')
as established by the ordering in the `ORDER BY` clause. The result set is sorted:
```
SELECT name, regionkey
FROM nation
ORDER BY regionkey FETCH FIRST ROW WITH TIES;
```
```sql theme={null}
name | regionkey
------------+-----------
ETHIOPIA | 0
MOROCCO | 0
KENYA | 0
ALGERIA | 0
MOZAMBIQUE | 0
(5 rows)
```
## TABLESAMPLE
There are multiple sample methods:
`BERNOULLI`
: Each row is selected to be in the table sample with a probability of
the sample percentage. When a table is sampled using the Bernoulli
method, all physical blocks of the table are scanned and certain
rows are skipped (based on a comparison between the sample percentage
and a random value calculated at runtime).
The probability of a row being included in the result is independent
from any other row. This does not reduce the time required to read
the sampled table from disk. It may have an impact on the total
query time if the sampled output is processed further.
`SYSTEM`
: This sampling method divides the table into logical segments of data
and samples the table at this granularity. This sampling method either
selects all the rows from a particular segment of data or skips it
(based on a comparison between the sample percentage and a random
value calculated at runtime).
The rows selected in a system sampling will be dependent on which
connector is used. For example, when used with Hive, it is dependent
on how the data is laid out on HDFS. This method does not guarantee
independent sampling probabilities.
Neither of the two methods allow deterministic bounds on the number of rows returned.
Examples:
```sql theme={null}
SELECT *
FROM users TABLESAMPLE BERNOULLI (50);
SELECT *
FROM users TABLESAMPLE SYSTEM (75);
```
Using sampling with joins:
```sql theme={null}
SELECT o.*, i.*
FROM orders o TABLESAMPLE SYSTEM (10)
JOIN lineitem i TABLESAMPLE BERNOULLI (40)
ON o.orderkey = i.orderkey;
```
## UNNEST
```sql theme={null}
SELECT * FROM UNNEST(ARRAY[1,2]) AS t(number);
```
```sql theme={null}
number
--------
1
2
(2 rows)
```
Maps are expanded into two columns (key, value):
```sql theme={null}
SELECT * FROM UNNEST(
map_from_entries(
ARRAY[
('SQL',1974),
('Java', 1995)
]
)
) AS t(language, first_appeared_year);
```
```sql theme={null}
language | first_appeared_year
----------+---------------------
SQL | 1974
Java | 1995
(2 rows)
```
```sql theme={null}
SELECT *
FROM UNNEST(
ARRAY[
ROW('Java', 1995),
ROW('SQL' , 1974)],
ARRAY[
ROW(false),
ROW(true)]
) as t(language,first_appeared_year,declarative);
```
```sql theme={null}
language | first_appeared_year | declarative
----------+---------------------+-------------
Java | 1995 | false
SQL | 1974 | true
(2 rows)
```
`UNNEST` can optionally have a `WITH ORDINALITY` clause, in which case an additional ordinality column
is added to the end:
```sql theme={null}
SELECT a, b, rownumber
FROM UNNEST (
ARRAY[2, 5],
ARRAY[7, 8, 9]
) WITH ORDINALITY AS t(a, b, rownumber);
```
```sql theme={null}
a | b | rownumber
------+---+-----------
2 | 7 | 1
5 | 8 | 2
NULL | 9 | 3
(3 rows)
```
`UNNEST` returns zero entries when the array/map is empty:
```sql theme={null}
SELECT * FROM UNNEST (ARRAY[]) AS t(value);
```
```sql theme={null}
value
-------
(0 rows)
```
`UNNEST` returns zero entries when the array/map is null:
```sql theme={null}
SELECT * FROM UNNEST (CAST(null AS ARRAY(integer))) AS t(number);
```
```sql theme={null}
number
--------
(0 rows)
```
`UNNEST` is normally used with a `JOIN`, and can reference columns
from relations on the left side of the join:
```sql theme={null}
SELECT student, score
FROM (
VALUES
('John', ARRAY[7, 10, 9]),
('Mary', ARRAY[4, 8, 9])
) AS tests (student, scores)
CROSS JOIN UNNEST(scores) AS t(score);
```
```sql theme={null}
student | score
---------+-------
John | 7
John | 10
John | 9
Mary | 4
Mary | 8
Mary | 9
(6 rows)
```
`UNNEST` can also be used with multiple arguments, in which case they are expanded into multiple columns,
with as many rows as the highest cardinality argument (the other columns are padded with nulls):
```sql theme={null}
SELECT numbers, animals, n, a
FROM (
VALUES
(ARRAY[2, 5], ARRAY['dog', 'cat', 'bird']),
(ARRAY[7, 8, 9], ARRAY['cow', 'pig'])
) AS x (numbers, animals)
CROSS JOIN UNNEST(numbers, animals) AS t (n, a);
```
```sql theme={null}
numbers | animals | n | a
-----------+------------------+------+------
[2, 5] | [dog, cat, bird] | 2 | dog
[2, 5] | [dog, cat, bird] | 5 | cat
[2, 5] | [dog, cat, bird] | NULL | bird
[7, 8, 9] | [cow, pig] | 7 | cow
[7, 8, 9] | [cow, pig] | 8 | pig
[7, 8, 9] | [cow, pig] | 9 | NULL
(6 rows)
```
`LEFT JOIN` is preferable in order to avoid losing the the row containing the array/map field in question
when referenced columns from relations on the left side of the join can be empty or have `NULL` values:
```sql theme={null}
SELECT runner, checkpoint
FROM (
VALUES
('Joe', ARRAY[10, 20, 30, 42]),
('Roger', ARRAY[10]),
('Dave', ARRAY[]),
('Levi', NULL)
) AS marathon (runner, checkpoints)
LEFT JOIN UNNEST(checkpoints) AS t(checkpoint) ON TRUE;
```
```sql theme={null}
runner | checkpoint
--------+------------
Joe | 10
Joe | 20
Joe | 30
Joe | 42
Roger | 10
Dave | NULL
Levi | NULL
(7 rows)
```
Note that in case of using `LEFT JOIN` the only condition supported by the current implementation is `ON TRUE`.
## Joins
Joins allow you to combine data from multiple relations.
### CROSS JOIN
A cross join returns the Cartesian product (all combinations) of two
relations. Cross joins can either be specified using the explit
`CROSS JOIN` syntax or by specifying multiple relations in the
`FROM` clause.
Both of the following queries are equivalent:
```sql theme={null}
SELECT *
FROM nation
CROSS JOIN region;
SELECT *
FROM nation, region;
```
The `nation` table contains 25 rows and the `region` table contains 5 rows,
so a cross join between the two tables produces 125 rows:
```sql theme={null}
SELECT n.name AS nation, r.name AS region
FROM nation AS n
CROSS JOIN region AS r
ORDER BY 1, 2;
```
```sql theme={null}
nation | region
----------------+-------------
ALGERIA | AFRICA
ALGERIA | AMERICA
ALGERIA | ASIA
ALGERIA | EUROPE
ALGERIA | MIDDLE EAST
ARGENTINA | AFRICA
ARGENTINA | AMERICA
...
(125 rows)
```
### LATERAL
Subqueries appearing in the `FROM` clause can be preceded by the keyword `LATERAL`.
This allows them to reference columns provided by preceding `FROM` items.
A `LATERAL` join can appear at the top level in the `FROM` list, or anywhere
within a parenthesized join tree. In the latter case, it can also refer to any items
that are on the left-hand side of a `JOIN` for which it is on the right-hand side.
When a `FROM` item contains `LATERAL` cross-references, evaluation proceeds as follows:
for each row of the `FROM` item providing the cross-referenced columns,
the `LATERAL` item is evaluated using that row set's values of the columns.
The resulting rows are joined as usual with the rows they were computed from.
This is repeated for set of rows from the column source tables.
`LATERAL` is primarily useful when the cross-referenced column is necessary for
computing the rows to be joined:
```sql theme={null}
SELECT name, x, y
FROM nation
CROSS JOIN LATERAL (SELECT name || ' :-' AS x)
CROSS JOIN LATERAL (SELECT x || ')' AS y);
```
### Qualifying column names
When two relations in a join have columns with the same name, the column
references must be qualified using the relation alias (if the relation
has an alias), or with the relation name:
```sql theme={null}
SELECT nation.name, region.name
FROM nation
CROSS JOIN region;
SELECT n.name, r.name
FROM nation AS n
CROSS JOIN region AS r;
SELECT n.name, r.name
FROM nation n
CROSS JOIN region r;
```
The following query will fail with the error `Column 'name' is ambiguous`:
```sql theme={null}
SELECT name
FROM nation
CROSS JOIN region;
```
## Subqueries
A subquery is an expression which is composed of a query. The subquery
is correlated when it refers to columns outside of the subquery.
Logically, the subquery will be evaluated for each row in the surrounding
query. The referenced columns will thus be constant during any single
evaluation of the subquery.
Support for correlated subqueries is limited. Not every standard form is supported.
### EXISTS
The `EXISTS` predicate determines if a subquery returns any rows:
```sql theme={null}
SELECT name
FROM nation
WHERE EXISTS (
SELECT *
FROM region
WHERE region.regionkey = nation.regionkey
);
```
### IN
The `IN` predicate determines if any values produced by the subquery
are equal to the provided expression. The result of `IN` follows the
standard rules for nulls. The subquery must produce exactly one column:
```sql theme={null}
SELECT name
FROM nation
WHERE regionkey IN (
SELECT regionkey
FROM region
WHERE name = 'AMERICA' OR name = 'AFRICA'
);
```
### Scalar subquery
A scalar subquery is a non-correlated subquery that returns zero or
one row. It is an error for the subquery to produce more than one
row. The returned value is `NULL` if the subquery produces no rows:
```sql theme={null}
SELECT name
FROM nation
WHERE regionkey = (SELECT max(regionkey) FROM region);
```
Currently only single column can be returned from the scalar subquery.
# SHOW CATALOGS
Source: https://docs.upsolve.ai/sql/reference/show-catalogs
## Synopsis
```sql theme={null}
SHOW CATALOGS [ LIKE pattern ]
```
## Description
List the available catalogs.
[Specify a pattern](/sql/functions/comparison#pattern-comparison-like) in the optional `LIKE` clause to
filter the results to the desired subset. For example, the following query
allows you to find catalogs that begin with `t`:
```sql theme={null}
SHOW CATALOGS LIKE 't%'
```
# SHOW COLUMNS
Source: https://docs.upsolve.ai/sql/reference/show-columns
## Synopsis
```text theme={null}
SHOW COLUMNS FROM table [ LIKE pattern ]
```
## Description
List the columns in a `table` along with their data type and other attributes:
```sql theme={null}
SHOW COLUMNS FROM nation;
```
```sql theme={null}
Column | Type | Extra | Comment
-----------+--------------+-------+---------
nationkey | bigint | |
name | varchar(25) | |
regionkey | bigint | |
comment | varchar(152) | |
```
[Specify a pattern](/sql/functions/comparison#pattern-comparison-like) in the optional `LIKE` clause to
filter the results to the desired subset. For example, the following query
allows you to find columns ending in `key`:
```sql theme={null}
SHOW COLUMNS FROM nation LIKE '%key';
```
```sql theme={null}
Column | Type | Extra | Comment
-----------+--------------+-------+---------
nationkey | bigint | |
regionkey | bigint | |
```
# SHOW SCHEMAS
Source: https://docs.upsolve.ai/sql/reference/show-schemas
## Synopsis
```sql theme={null}
SHOW SCHEMAS [ FROM catalog ] [ LIKE pattern ]
```
## Description
List the schemas in `catalog` or in the current catalog.
[Specify a pattern](/sql/functions/comparison#pattern-comparison-like) in the optional `LIKE` clause to
filter the results to the desired subset. For example, the following query
allows you to find schemas that have `3` as the third character:
```sql theme={null}
SHOW SCHEMAS FROM tpch LIKE '__3%'
```
# Data types
Source: https://docs.upsolve.ai/sql/reference/types
Upsolve has a set of built-in data types, described below.
## Upsolve type support and mapping
Connectors to data sources are not required to support all Upsolve data types
described on this page. If there are data types similar to Upsolve's that are used
on the data source, the connector may map the Upsolve and remote data types to
each other as needed.
Depending on the connector and the data source, type mapping may apply
in either direction as follows:
* **Data source to Upsolve** mapping applies to any operation where columns in the
data source are read by Upsolve, such as a [SELECT](/sql/reference/select) statement, and the
underlying source data type needs to be represented by a Upsolve data type.
## Boolean
### `BOOLEAN`
This type captures boolean values `true` and `false`.
## Integer
Integer numbers can be expressed as numeric literals in the following formats:
* Decimal integer. Examples are `-7`, `0`, or `3`.
* Hexadecimal integer composed of `0X` or `0x` and the value. Examples are
`0x0A` for decimal `10` or `0x11` for decimal `17`.
* Octal integer composed of `0O` or `0o` and the value. Examples are `0o40` for
decimal `32` or `0o11` for decimal `9`.
* Binary integer composed of `0B` or `0b` and the value. Examples are `0b1001`
for decimal `9` or `0b101010` for decimal \`42\`\`.
Underscore characters are ignored within literal values, and can be used to
increase readability. For example, decimal integer `123_456.789_123` is
equivalent to `123456.789123`. Preceding and trailing underscores are not
permitted.
Integers are supported by the following data types.
### `TINYINT`
A 8-bit signed two's complement integer with a minimum value of
`-2^7` or `-0x80` and a maximum value of `2^7 - 1` or `0x7F`.
### `SMALLINT`
A 16-bit signed two's complement integer with a minimum value of
`-2^15` or `-0x8000` and a maximum value of `2^15 - 1` or `0x7FFF`.
### `INTEGER` or `INT`
A 32-bit signed two's complement integer with a minimum value of `-2^31` or
`-0x80000000` and a maximum value of `2^31 - 1` or `0x7FFFFFFF`. The names
`INTEGER` and `INT` can both be used for this type.
### `BIGINT`
A 64-bit signed two's complement integer with a minimum value of `-2^63` or
`-0x8000000000000000` and a maximum value of `2^63 - 1` or `0x7FFFFFFFFFFFFFFF`.
## Floating-point
Floating-point, fixed-precision numbers can be expressed as numeric literal
using scientific notation such as `1.03e1` and are cast as `DOUBLE` data type.
Underscore characters are ignored within literal values, and can be used to
increase readability. For example, value `123_456.789e4` is equivalent to
`123456.789e4`. Preceding underscores, trailing underscores, and underscores
beside the comma (`.`) are not permitted.
### `REAL`
A real is a 32-bit inexact, variable-precision implementing the
IEEE Standard 754 for Binary Floating-Point Arithmetic.
Example literals: `REAL '10.3'`, `REAL '10.3e0'`, `REAL '1.03e1'`
### `DOUBLE`
A double is a 64-bit inexact, variable-precision implementing the
IEEE Standard 754 for Binary Floating-Point Arithmetic.
Example literals: `DOUBLE '10.3'`, `DOUBLE '1.03e1'`, `10.3e0`, `1.03e1`
## Exact numeric
Exact numeric values can be expressed as numeric literals such as `1.1`, and
are supported by the `DECIMAL` data type.
Underscore characters are ignored within literal values, and can be used to
increase readability. For example, decimal `123_456.789_123` is equivalent to
`123456.789123`. Preceding underscores, trailing underscores, and underscores
beside the comma (`.`) are not permitted.
Leading zeros in literal values are permitted and ignored. For example,
`000123.456` is equivalent to `123.456`.
### `DECIMAL`
A exact decimal number. Precision up to 38 digits is supported but performance
is best up to 18 digits.
The decimal type takes two literal parameters:
* **precision** - total number of digits
* **scale** - number of digits in fractional part. Scale is optional and defaults to 0.
Example type definitions: `DECIMAL(10,3)`, `DECIMAL(20)`
Example literals: `DECIMAL '10.3'`, `DECIMAL '1234567890'`, `1.1`
## String
### `VARCHAR`
Variable length character data with an optional maximum length.
Example type definitions: `varchar`, `varchar(20)`
SQL statements support simple literal, as well as Unicode usage:
* literal string : `'Hello winter !'`
* Unicode string with default escape character: `U&'Hello winter \2603 !'`
* Unicode string with custom escape character: `U&'Hello winter #2603 !' UESCAPE '#'`
A Unicode string is prefixed with `U&` and requires an escape character
before any Unicode character usage with 4 digits. In the examples above
`\2603` and `#2603` represent a snowman character. Long Unicode codes
with 6 digits require usage of the plus symbol before the code. For example,
you need to use `\+01F600` for a grinning face emoji.
Single quotes in string literals can be escaped by using another single quote:
`'I am big, it''s the pictures that got small!'`
### `CHAR`
Fixed length character data. A `CHAR` type without length specified has a default length of 1.
A `CHAR(x)` value always has `x` characters. For example, casting `dog` to `CHAR(7)`
adds 4 implicit trailing spaces. Leading and trailing spaces are included in comparisons of
`CHAR` values. As a result, two character values with different lengths (`CHAR(x)` and
`CHAR(y)` where `x != y`) will never be equal. As with `VARCHAR`, a single quote in a `CHAR`
literal can be escaped with another single quote:
```sql theme={null}
SELECT CHAR 'All right, Mr. DeMille, I''m ready for my close-up.'
```
Example type definitions: `char`, `char(20)`
### `VARBINARY`
Variable length binary data.
SQL statements support usage of binary literal data with the prefix `X` or `x`.
The binary data has to use hexadecimal format. For example, the binary form of
`eh?` is `X'65683F'` as you can confirm with the following statement:
```sql theme={null}
SELECT from_utf8(x'65683F');
```
Binary strings with length are not yet supported: `varbinary(n)`
### `JSON`
JSON value type, which can be a JSON object, a JSON array, a JSON number, a JSON string,
`true`, `false` or `null`.
## Date and time
See also [Date and Time functions](/sql/functions/datetime)
### `DATE`
Calendar date (year, month, day).
Example: `DATE '2001-08-22'`
### `TIME`
`TIME` is an alias for `TIME(3)` (millisecond precision).
### `TIME(P)`
Time of day (hour, minute, second) without a time zone with `P` digits of precision
for the fraction of seconds. A precision of up to 12 (picoseconds) is supported.
Example: `TIME '01:02:03.456'`
### `TIME WITH TIME ZONE`
Time of day (hour, minute, second, millisecond) with a time zone.
Values of this type are rendered using the time zone from the value.
Time zones are expressed as the numeric UTC offset value:
```
SELECT TIME '01:02:03.456 -08:00';
-- 1:02:03.456-08:00
```
### `TIMESTAMP`
`TIMESTAMP` is an alias for `TIMESTAMP(3)` (millisecond precision).
### `TIMESTAMP(P)`
Calendar date and time of day without a time zone with `P` digits of precision
for the fraction of seconds. A precision of up to 12 (picoseconds) is supported.
This type is effectively a combination of the `DATE` and `TIME(P)` types.
`TIMESTAMP(P) WITHOUT TIME ZONE` is an equivalent name.
Timestamp values can be constructed with the `TIMESTAMP` literal
expression. Alternatively, language constructs such as
`localtimestamp(p)`, or a number of [date and time functions and
operators](/sql/functions/datetime) can return timestamp values.
Casting to lower precision causes the value to be rounded, and not
truncated. Casting to higher precision appends zeros for the additional
digits.
The following examples illustrate the behavior:
```
SELECT TIMESTAMP '2020-06-10 15:55:23';
-- 2020-06-10 15:55:23
SELECT TIMESTAMP '2020-06-10 15:55:23.383345';
-- 2020-06-10 15:55:23.383345
SELECT typeof(TIMESTAMP '2020-06-10 15:55:23.383345');
-- timestamp(6)
SELECT cast(TIMESTAMP '2020-06-10 15:55:23.383345' as TIMESTAMP(1));
-- 2020-06-10 15:55:23.4
SELECT cast(TIMESTAMP '2020-06-10 15:55:23.383345' as TIMESTAMP(12));
-- 2020-06-10 15:55:23.383345000000
```
### `TIMESTAMP WITH TIME ZONE`
`TIMESTAMP WITH TIME ZONE` is an alias for `TIMESTAMP(3) WITH TIME ZONE`
(millisecond precision).
### `TIMESTAMP(P) WITH TIME ZONE`
Instant in time that includes the date and time of day with `P` digits of
precision for the fraction of seconds and with a time zone. Values of this type
are rendered using the time zone from the value. Time zones can be expressed in
the following ways:
* `UTC`, with `GMT`, `Z`, or `UT` usable as aliases for UTC.
* `+hh:mm` or `-hh:mm` with `hh:mm` as an hour and minute offset from UTC.
Can be written with or without `UTC`, `GMT`, or `UT` as an alias for
UTC.
* An [IANA time zone name](https://www.iana.org/time-zones).
The following examples demonstrate some of these syntax options:
```
SELECT TIMESTAMP '2001-08-22 03:04:05.321 UTC';
-- 2001-08-22 03:04:05.321 UTC
SELECT TIMESTAMP '2001-08-22 03:04:05.321 -08:30';
-- 2001-08-22 03:04:05.321 -08:30
SELECT TIMESTAMP '2001-08-22 03:04:05.321 GMT-08:30';
-- 2001-08-22 03:04:05.321 -08:30
SELECT TIMESTAMP '2001-08-22 03:04:05.321 America/New_York';
-- 2001-08-22 03:04:05.321 America/New_York
```
### `INTERVAL YEAR TO MONTH`
Span of years and months.
Example: `INTERVAL '3' MONTH`
### `INTERVAL DAY TO SECOND`
Span of days, hours, minutes, seconds and milliseconds.
Example: `INTERVAL '2' DAY`
## Structural
### `ARRAY`
An array of the given component type.
Example: `ARRAY[1, 2, 3]`
### `MAP`
A map between the given component types.
Example: `MAP(ARRAY['foo', 'bar'], ARRAY[1, 2])`
### `ROW`
A structure made up of fields that allows mixed types.
The fields may be of any SQL type.
By default, row fields are not named, but names can be assigned.
Example: `CAST(ROW(1, 2e0) AS ROW(x BIGINT, y DOUBLE))`
Named row fields are accessed with field reference operator (`.`).
Example: `CAST(ROW(1, 2.0) AS ROW(x BIGINT, y DOUBLE)).x`
Named or unnamed row fields are accessed by position with the subscript
operator (`[]`). The position starts at `1` and must be a constant.
Example: `ROW(1, 2.0)[1]`
## Network address
### `IPADDRESS`
An IP address that can represent either an IPv4 or IPv6 address. Internally,
the type is a pure IPv6 address. Support for IPv4 is handled using the
*IPv4-mapped IPv6 address* range [RFC 4291#section-2.5.5.2](https://datatracker.ietf.org/doc/html/rfc4291.html#section-2.5.5.2`).
When creating an `IPADDRESS`, IPv4 addresses will be mapped into that range.
When formatting an `IPADDRESS`, any address within the mapped range will
be formatted as an IPv4 address. Other addresses will be formatted as IPv6
using the canonical format defined in [RFC 5952](https://datatracker.ietf.org/doc/html/rfc5952.html).
Examples: `IPADDRESS '10.0.0.1'`, `IPADDRESS '2001:db8::1'`
## UUID
### `UUID`
This type represents a UUID (Universally Unique IDentifier), also known as a
GUID (Globally Unique IDentifier), using the format defined in [RFC 4122](https://datatracker.ietf.org/doc/html/rfc4122.html).
Example: `UUID '12151fd2-7586-11e9-8f9e-2a86e4085a59'`
## HyperLogLog
Calculating the approximate distinct count can be done much more cheaply than an exact count using the
[HyperLogLog](https://wikipedia.org/wiki/HyperLogLog) data sketch. See `/functions/hyperloglog`.
### `HyperLogLog`
A HyperLogLog sketch allows efficient computation of [approx\_distinct()](/sql/functions/aggregate#approx-distinct). It starts as a
sparse representation, switching to a dense representation when it becomes more efficient.
### `P4HyperLogLog`
A P4HyperLogLog sketch is similar to [HyperLogLog](/sql/reference/types#hyperloglog), but it starts (and remains)
in the dense representation.
## SetDigest
### `SetDigest`
A SetDigest (setdigest) is a data sketch structure used
in calculating [Jaccard similarity coefficient](https://wikipedia.org/wiki/Jaccard_index)
between two sets.
SetDigest encapsulates the following components:
* [HyperLogLog](https://wikipedia.org/wiki/HyperLogLog)
* [MinHash with a single hash function](http://wikipedia.org/wiki/MinHash#Variant_with_a_single_hash_function)
The HyperLogLog structure is used for the approximation of the distinct elements
in the original set.
The MinHash structure is used to store a low memory footprint signature of the original set.
The similarity of any two sets is estimated by comparing their signatures.
SetDigests are additive, meaning they can be merged together.
## Quantile digest
### `QDigest`
A quantile digest (qdigest) is a summary structure which captures the approximate
distribution of data for a given input set, and can be queried to retrieve approximate
quantile values from the distribution. The level of accuracy for a qdigest
is tunable, allowing for more precise results at the expense of space.
A qdigest can be used to give approximate answer to queries asking for what value
belongs at a certain quantile. A useful property of qdigests is that they are
additive, meaning they can be merged together without losing precision.
A qdigest may be helpful whenever the partial results of `approx_percentile`
can be reused. For example, one may be interested in a daily reading of the 99th
percentile values that are read over the course of a week. Instead of calculating
the past week of data with `approx_percentile`, `qdigest`s could be stored
daily, and quickly merged to retrieve the 99th percentile value.
## T-Digest
### `TDigest`
A T-digest (tdigest) is a summary structure which, similarly to qdigest, captures the
approximate distribution of data for a given input set. It can be queried to retrieve
approximate quantile values from the distribution.
TDigest has the following advantages compared to QDigest:
* higher performance
* lower memory usage
* higher accuracy at high and low percentiles
T-digests are additive, meaning they can be merged together.
# USE
Source: https://docs.upsolve.ai/sql/reference/use
## Synopsis
```sql theme={null}
USE catalog.schema
USE schema
```
## Description
Update the session to use the specified catalog and schema. If a
catalog is not specified, the schema is resolved relative to the
current catalog.
## Examples
```sql theme={null}
USE hive.finance;
USE information_schema;
```
# VALUES
Source: https://docs.upsolve.ai/sql/reference/values
## Synopsis
```sql theme={null}
VALUES row [, ...]
```
where `row` is a single expression or
```sql theme={null}
( column_expression [, ...] )
```
## Description
Defines a literal inline table.
`VALUES` can be used anywhere a query can be used (e.g., the `FROM` clause
of a [SELECT](/sql/reference/select), or at the top level). `VALUES` creates
an anonymous table without column names, but the table and columns can be named
using an `AS` clause with column aliases.
## Examples
Return a table with one column and three rows:
```sql theme={null}
VALUES 1, 2, 3
```
Return a table with two columns and three rows:
```sql theme={null}
VALUES
(1, 'a'),
(2, 'b'),
(3, 'c')
```
Return table with column `id` and `name`:
```sql theme={null}
SELECT * FROM (
VALUES
(1, 'a'),
(2, 'b'),
(3, 'c')
) AS t (id, name)
```
Create a new table with column `id` and `name`:
```sql theme={null}
CREATE TABLE example AS
SELECT * FROM (
VALUES
(1, 'a'),
(2, 'b'),
(3, 'c')
) AS t (id, name)
```
## See also
[SELECT](/sql/reference/select)
# BEGIN
Source: https://docs.upsolve.ai/sql/routines/begin
## Synopsis
```text theme={null}
BEGIN
[ DECLARE ... ]
statements
END
```
## Description
Marks the start and end of a block in a [SQL routine](/sql/routines/introduction).
`BEGIN` can be used wherever a statement can be used to group multiple
statements together and to declare variables local to the block. A typical use
case is as first statement within a [function](/sql/routines/function). Blocks can also be
nested.
After the `BEGIN` keyword, you can add variable declarations using
[declare](/sql/routines/declare) statements, followed by one or more statements that define
the main body of the routine, separated by `;`. The following statements can be
used:
* [case](/sql/routines/case)
* [if](/sql/routines/if)
* [iterate](/sql/routines/iterate)
* [leave](/sql/routines/leave)
* [loop](/sql/routines/loop)
* [repeat](/sql/routines/repeat)
* [return](/sql/routines/return)
* [set](/sql/routines/set)
* [while](/sql/routines/while)
* Nested [begin](/sql/routines/begin) blocks
## Examples
The following example computes the value `42`:
```sql theme={null}
FUNCTION meaning_of_life()
RETURNS tinyint
BEGIN
DECLARE a tinyint DEFAULT 6;
DECLARE b tinyint DEFAULT 7;
RETURN a * b;
END
```
Further examples of varying complexity that cover usage of the `BEGIN` statement
in combination with other statements are available in the [SQL routines examples
documentation](/sql/routines/examples).
## See also
* [routines](/sql/routines/introduction)
* [function](/sql/routines/function)
# CASE
Source: https://docs.upsolve.ai/sql/routines/case
## Synopsis
Simple case:
```text theme={null}
CASE
WHEN condition THEN statements
[ ... ]
[ ELSE statements ]
END CASE
```
Searched case:
```text theme={null}
CASE expression
WHEN expression THEN statements
[ ... ]
[ ELSE statements ]
END
```
## Description
The `CASE` statement is an optional construct to allow conditional processing
in [SQL routines](/sql/routines/introduction).
The `WHEN` clauses are evaluated sequentially, stopping after the first match,
and therefore the order of the statements is significant. The statements of the
`ELSE` clause are executed if none of the `WHEN` clauses match.
Unlike other languages like C or Java, SQL does not support case fall through,
so processing stops at the end of the first matched case.
One or more `WHEN` clauses can be used.
## Examples
The following example shows a simple `CASE` statement usage:
```sql theme={null}
FUNCTION simple_case(a bigint)
RETURNS varchar
BEGIN
CASE a
WHEN 0 THEN RETURN 'zero';
WHEN 1 THEN RETURN 'one';
ELSE RETURN 'more than one or negative';
END CASE;
RETURN NULL;
END
```
Further examples of varying complexity that cover usage of the `CASE` statement
in combination with other statements are available in the [SQL routines examples
documentation](/sql/routines/examples).
## See also
* [routines](/sql/routines/introduction)
* [Conditional expressions using `CASE`](/sql/functions/conditional#case-expression)
# DECLARE
Source: https://docs.upsolve.ai/sql/routines/declare
## Synopsis
```text theme={null}
DECLARE identifier [, ...] type [ DEFAULT expression ]
```
## Description
Use the `DECLARE` statement directly after the [begin](/sql/routines/begin) keyword in
[routines](/sql/routines/introduction) to define one or more variables with an `identifier` as name.
## Examples
A simple declaration of the variable `x` with the `tinyint` data type and the
implicit default value of `null`:
```sql theme={null}
DECLARE x tinyint;
```
A declaration of multiple string variables with length restricted to 25
characters:
```sql theme={null}
DECLARE first_name, last_name, middle_name varchar(25);
```
A declaration of an exact decimal number with a default value:
```sql theme={null}
DECLARE uptime_requirement decimal DEFAULT 99.999;
```
A declaration with a default value from an expression:
```sql theme={null}
DECLARE start_time timestamp(3) with time zone DEFAULT now();
```
Further examples of varying complexity that cover usage of the `DECLARE`
statement in combination with other statements are available in the [SQL
routines examples documentation](/sql/routines/examples).
## See also
* [routines](/sql/routines/introduction)
# Example SQL routines
Source: https://docs.upsolve.ai/sql/routines/examples
After learning about [SQL routines from the
introduction](/sql/routines/introduction), the following sections show numerous
examples of valid SQL routines. The routines are suitable as [inline
routines](/sql/routines/introduction#inline-routines) or [catalog routines](/sql/routines/introduction#catalog-routines), after
adjusting the name and adjusting the example invocations.
The examples combine numerous supported statements. Refer to the specific
statement documentation for further details:
* [function](/sql/routines/function) for general SQL routine declaration
* [begin](/sql/routines/begin) and [declare](/sql/routines/declare) for routine blocks
* [set](/sql/routines/set) for assigning values to variables
* [return](/sql/routines/return) for returning routine results
* [case](/sql/routines/case) and [if](/sql/routines/if) for conditional flows
* [loop](/sql/routines/loop), [repeat](/sql/routines/repeat), and [while](/sql/routines/while) for looping constructs
* [iterate](/sql/routines/iterate) and [leave](/sql/routines/leave) for flow control
A very simple routine that returns a static value without requiring any input:
```sql theme={null}
FUNCTION answer()
RETURNS BIGINT
RETURN 42
```
## Inline and catalog routines
A full example of this routine as inline routine and usage in a string
concatenation with a cast:
```sql theme={null}
WITH
FUNCTION answer()
RETURNS BIGINT
RETURN 42
SELECT 'The answer is ' || CAST(answer() as varchar);
-- The answer is 42
```
Provided the catalog `example` supports routine storage in the `default` schema,
you can use the following:
```sql theme={null}
USE example.default;
CREATE FUNCTION example.default.answer()
RETURNS BIGINT
RETURN 42;
```
With the routine stored in the catalog, you can run the routine multiple times
without repeated definition:
```sql theme={null}
SELECT example.default.answer() + 1; -- 43
SELECT 'The answer is' || CAST(example.default.answer() as varchar); -- The answer is 42
```
## Declaration examples
The result of calling the routine `answer()` is always identical, so you can
declare it as deterministic, and add some other information:
```sql theme={null}
FUNCTION answer()
LANGUAGE SQL
DETERMINISTIC
RETURNS BIGINT
COMMENT 'Provide the answer to the question about life, the universe, and everything.'
RETURN 42
```
A simple routine that returns a greeting back to the input string `fullname`
concatenating two strings and the input value:
```sql theme={null}
FUNCTION hello(fullname VARCHAR)
RETURNS VARCHAR
RETURN 'Hello, ' || fullname || '!'
```
Following is an example invocation:
```sql theme={null}
SELECT hello('Jane Doe'); -- Hello, Jane Doe!
```
A first example routine, that uses multiple statements in a `BEGIN` block. It
calculates the result of a multiplication of the input integer with `99`. The
`bigint` data type is used for all variables and values. The value of integer
`99` is cast to `bigint` in the default value assignment for the variable `x`.
```sql theme={null}
FUNCTION times_ninety_nine(a bigint)
RETURNS bigint
BEGIN
DECLARE x bigint DEFAULT CAST(99 AS bigint);
RETURN x * a;
END
```
Following is an example invocation:
```sql theme={null}
SELECT times_ninety_nine(CAST(2 as bigint)); -- 198
```
## Conditional flows
A first example of conditional flow control in a routine using the `CASE`
statement. The simple `bigint` input value is compared to a number of values.
```sql theme={null}
FUNCTION simple_case(a bigint)
RETURNS varchar
BEGIN
CASE a
WHEN 0 THEN RETURN 'zero';
WHEN 1 THEN RETURN 'one';
WHEN 10 THEN RETURN 'ten';
WHEN 20 THEN RETURN 'twenty';
ELSE RETURN 'other';
END CASE;
RETURN NULL;
END
```
Following are a couple of example invocations with result and explanation:
```sql theme={null}
SELECT simple_case(0); -- zero
SELECT simple_case(1); -- one
SELECT simple_case(-1); -- other (from else clause)
SELECT simple_case(10); -- ten
SELECT simple_case(11); -- other (from else clause)
SELECT simple_case(20); -- twenty
SELECT simple_case(100); -- other (from else clause)
SELECT simple_case(null); -- null .. but really??
```
A second example of a routine with a `CASE` statement, this time with two
parameters, showcasing the importance of the order of the conditions.
```sql theme={null}
FUNCTION search_case(a bigint, b bigint)
RETURNS varchar
BEGIN
CASE
WHEN a = 0 THEN RETURN 'zero';
WHEN b = 1 THEN RETURN 'one';
WHEN a = DECIMAL '10.0' THEN RETURN 'ten';
WHEN b = 20.0E0 THEN RETURN 'twenty';
ELSE RETURN 'other';
END CASE;
RETURN NULL;
END
```
Following are a couple of example invocations with result and explanation:
```sql theme={null}
SELECT search_case(0,0); -- zero
SELECT search_case(1,1); -- one
SELECT search_case(0,1); -- zero (not one since the second check is never reached)
SELECT search_case(10,1); -- one (not ten since the third check is never reached)
SELECT search_case(10,2); -- ten
SELECT search_case(10,20); -- ten (not twenty)
SELECT search_case(0,20); -- zero (not twenty)
SELECT search_case(3,20); -- twenty
SELECT search_case(3,21); -- other
SELECT simple_case(null,null); -- null .. but really??
```
## Fibonacci example
This routine calculates the `n`-th value in the Fibonacci series, in which each
number is the sum of the two preceding ones. The two initial values are set
to `1` as the defaults for `a` and `b`. The routine uses an `IF` statement
condition to return `1` for all input values of `2` or less. The `WHILE` block
then starts to calculate each number in the series, starting with `a=1` and
`b=1` and iterates until it reaches the `n`-th position. In each iteration is
sets `a` and `b` for the preceding to values, so it can calculate the sum, and
finally return it. Note that processing the routine takes longer and longer with
higher `n` values, and the result is deterministic.
```sql theme={null}
FUNCTION fib(n bigint)
RETURNS bigint
BEGIN
DECLARE a, b bigint DEFAULT 1;
DECLARE c bigint;
IF n <= 2 THEN
RETURN 1;
END IF;
WHILE n > 2 DO
SET n = n - 1;
SET c = a + b;
SET a = b;
SET b = c;
END WHILE;
RETURN c;
END
```
Following are a couple of example invocations with result and explanation:
```sql theme={null}
SELECT fib(-1); -- 1
SELECT fib(0); -- 1
SELECT fib(1); -- 1
SELECT fib(2); -- 1
SELECT fib(3); -- 2
SELECT fib(4); -- 3
SELECT fib(5); -- 5
SELECT fib(6); -- 8
SELECT fib(7); -- 13
SELECT fib(8); -- 21
```
## Labels and loops
This routine uses the `top` label to name the `WHILE` block, and then controls
the flow with conditional statements, `ITERATE`, and `LEAVE`. For the values of
`a=1` and `a=2` in the first two iterations of the loop the `ITERATE` call moves
the flow up to `top` before `b` is ever increased. Then `b` is increased for the
values `a=3`, `a=4`, `a=5`, `a=6`, and `a=7`, resulting in `b=5`. The `LEAVE`
call then causes the exit of the block before a is increased further to `10` and
therefore the result of the routine is `5`.
```sql theme={null}
FUNCTION labels()
RETURNS bigint
BEGIN
DECLARE a, b int DEFAULT 0;
top: WHILE a < 10 DO
SET a = a + 1;
IF a < 3 THEN
ITERATE top;
END IF;
SET b = b + 1;
IF a > 6 THEN
LEAVE top;
END IF;
END WHILE;
RETURN b;
END
```
This routine implements calculating the `n` to the power of `p` by repeated
multiplication and keeping track of the number of multiplications performed.
Note that this routine does not return the correct `0` for `p=0` since the `top`
block is merely escaped and the value of `n` is returned. The same incorrect
behavior happens for negative values of `p`:
```sql theme={null}
FUNCTION power(n int, p int)
RETURNS int
BEGIN
DECLARE r int DEFAULT n;
top: LOOP
IF p <= 1 THEN
LEAVE top;
END IF;
SET r = r * n;
SET p = p - 1;
END LOOP;
RETURN r;
END
```
Following are a couple of example invocations with result and explanation:
```sql theme={null}
SELECT power(2, 2); -- 4
SELECT power(2, 8); -- 256
SELECT power(3, 3); -- 256
SELECT power(3, 0); -- 3, which is wrong
SELECT power(3, -2); -- 3, which is wrong
```
This routine returns `7` as a result of the increase of `b` in the loop from
`a=3` to `a=10`:
```sql theme={null}
FUNCTION test_repeat_continue()
RETURNS bigint
BEGIN
DECLARE a int DEFAULT 0;
DECLARE b int DEFAULT 0;
top: REPEAT
SET a = a + 1;
IF a <= 3 THEN
ITERATE top;
END IF;
SET b = b + 1;
UNTIL a >= 10
END REPEAT;
RETURN b;
END
```
This routine returns `2` and shows that labels can be repeated and label usage
within a block refers to the label of that block:
```sql theme={null}
FUNCTION test()
RETURNS int
BEGIN
DECLARE r int DEFAULT 0;
abc: LOOP
SET r = r + 1;
LEAVE abc;
END LOOP;
abc: LOOP
SET r = r + 1;
LEAVE abc;
END LOOP;
RETURN r;
END
```
## Routines and built-in functions
This routine show that multiple data types and built-in functions like
`length()` and `cardinality()` can be used in a routine. The two nested `BEGIN`
blocks also show how variable names are local within these blocks `x`, but the
global `r` from the top-level block can be accessed in the nested blocks:
```sql theme={null}
FUNCTION test()
RETURNS bigint
BEGIN
DECLARE r bigint DEFAULT 0;
BEGIN
DECLARE x varchar DEFAULT 'hello';
SET r = r + length(x);
END;
BEGIN
DECLARE x array(int) DEFAULT array[1, 2, 3];
SET r = r + cardinality(x);
END;
RETURN r;
END
```
## Optional parameter example
Routines can invoke other routines and other functions. The full signature of a
routine is composed of routine name and parameters, and determines the exact
routine to use. You can declare multiple routines with the same name, but with
different number of arguments or different argument types. One example use case
is to implement an optional parameter.
The following routine truncates a string to the specified length including three
dots at the end of the output:
```sql theme={null}
FUNCTION dots(input varchar, length integer)
RETURNS varchar
BEGIN
IF length(input) > length THEN
RETURN substring(input, 1, length-3) || '...';
END IF;
RETURN input;
END;
```
Following are example invocations and output:
```sql theme={null}
SELECT dots('A long string that will be shortened',15);
-- A long strin...
SELECT dots('A short string',15);
-- A short string
```
If you want to provide a routine with the same name, but without the parameter
for length, you can create another routine that invokes the preceding routine:
```sql theme={null}
FUNCTION dots(input varchar)
RETURNS varchar
RETURN dots(input, 15);
```
You can now use both routines. When the length parameter is omitted the default
value from the second declaration is used.
```sql theme={null}
SELECT dots('A long string that will be shortened',15);
-- A long strin...
SELECT dots('A long string that will be shortened');
-- A long strin...
SELECT dots('A long string that will be shortened',20);
-- A long string tha...
```
## Date string parsing example
This example routine parses a date string of type `VARCHAR` into `TIMESTAMP WITH
TIME ZONE`. Date strings are commonly represented by ISO 8601 standard, such as
`2023-12-01`, `2023-12-01T23`. Date strings are also often represented in the
`YYYYmmdd` and `YYYYmmddHH` format, such as `20230101` and `2023010123`. Hive
tables can use this format to represent day and hourly partitions, for example
`/day=20230101`, `/hour=2023010123`.
This routine parses date strings in a best-effort fashion and can be used as a
replacement for date string manipulation functions such as `date`, `date_parse`,
`from_iso8601_date`, and `from_iso8601_timestamp`.
Note that the routine defaults the time value to `00:00:00.000` and the time
zone to the session time zone.
```sql theme={null}
FUNCTION from_date_string(date_string VARCHAR)
RETURNS TIMESTAMP WITH TIME ZONE
BEGIN
IF date_string like '%-%' THEN -- ISO 8601
RETURN from_iso8601_timestamp(date_string);
ELSEIF length(date_string) = 8 THEN -- YYYYmmdd
RETURN date_parse(date_string, '%Y%m%d');
ELSEIF length(date_string) = 10 THEN -- YYYYmmddHH
RETURN date_parse(date_string, '%Y%m%d%H');
END IF;
RETURN NULL;
END
```
Following are a couple of example invocations with result and explanation:
```sql theme={null}
SELECT from_date_string('2023-01-01'); -- 2023-01-01 00:00:00.000 UTC (using the ISO 8601 format)
SELECT from_date_string('2023-01-01T23'); -- 2023-01-01 23:00:00.000 UTC (using the ISO 8601 format)
SELECT from_date_string('2023-01-01T23:23:23'); -- 2023-01-01 23:23:23.000 UTC (using the ISO 8601 format)
SELECT from_date_string('20230101'); -- 2023-01-01 00:00:00.000 UTC (using the YYYYmmdd format)
SELECT from_date_string('2023010123'); -- 2023-01-01 23:00:00.000 UTC (using the YYYYmmddHH format)
SELECT from_date_string(NULL); -- NULL (handles NULL string)
SELECT from_date_string('abc'); -- NULL (not matched to any format)
```
## Human readable days
Trino includes a built-in function called `human_readable_seconds` that
formats a number of seconds into a string:
```sql theme={null}
SELECT human_readable_seconds(134823);
-- 1 day, 13 hours, 27 minutes, 3 seconds
```
The example routine `hrd` formats a number of days into a human readable text
that provides the approximate number of years and months:
```sql theme={null}
FUNCTION hrd(d integer)
RETURNS VARCHAR
BEGIN
DECLARE answer varchar default 'About ';
DECLARE years real;
DECLARE months real;
SET years = truncate(d/365);
IF years > 0 then
SET answer = answer || format('%1.0f', years) || ' year';
END IF;
IF years > 1 THEN
SET answer = answer || 's';
END IF;
SET d = d - cast( years AS integer) * 365 ;
SET months = truncate(d / 30);
IF months > 0 and years > 0 THEN
SET answer = answer || ' and ';
END IF;
IF months > 0 THEN
set answer = answer || format('%1.0f', months) || ' month';
END IF;
IF months > 1 THEN
SET answer = answer || 's';
END IF;
IF years < 1 and months < 1 THEN
SET answer = 'Less than 1 month';
END IF;
RETURN answer;
END;
```
The following examples show the output for a range of values under one month, under
one year, and various larger values:
```sql theme={null}
SELECT hrd(10); -- Less than 1 month
SELECT hrd(95); -- About 3 months
SELECT hrd(400); -- About 1 year and 1 month
SELECT hrd(369); -- About 1 year
SELECT hrd(800); -- About 2 years and 2 months
SELECT hrd(1100); -- About 3 years
SELECT hrd(5000); -- About 13 years and 8 months
```
Improvements of the routine could include the following modifications:
* Take into account that one month equals 30.4375 days.
* Take into account that one year equals 365.25 days.
* Add weeks to the output.
* Expand the cover decades, centuries, and millenia.
## Truncating long strings
This example routine `strtrunc` truncates strings longer than 60 characters,
leaving the first 30 and the last 25 characters, and cutting out extra
characters in the middle.
```sql theme={null}
FUNCTION strtrunc(input VARCHAR)
RETURNS VARCHAR
RETURN
CASE WHEN length(input) > 60
THEN substr(input, 1, 30) || ' ... ' || substr(input, length(input) - 25)
ELSE input
END;
```
The preceding declaration is very compact and consists of only one complex
statement with a [`CASE` expression](/sql/functions/conditional#case-expression) and multiple function
calls. It can therefore define the complete logic in the `RETURN` clause.
The following statement shows the same capability within the routine itself.
Note the duplicate `RETURN` inside and outside the `CASE` statement and the
required `END CASE;`. The second `RETURN` statement is required, because a
routine must end with a `RETURN` statement. As a result the `ELSE` clause can be
omitted.
```sql theme={null}
FUNCTION strtrunc(input VARCHAR)
RETURNS VARCHAR
BEGIN
CASE WHEN length(input) > 60
THEN
RETURN substr(input, 1, 30) || ' ... ' || substr(input, length(input) - 25);
ELSE
RETURN input;
END CASE;
RETURN input;
END;
```
The next example changes over from a `CASE` to an `IF` statement, and avoids the
duplicate `RETURN`:
```sql theme={null}
FUNCTION strtrunc(input VARCHAR)
RETURNS VARCHAR
BEGIN
IF length(input) > 60 THEN
RETURN substr(input, 1, 30) || ' ... ' || substr(input, length(input) - 25);
END IF;
RETURN input;
END;
```
All the preceding examples create the same output. Following is an example query
which generates long strings to truncate:
```sql theme={null}
WITH
data AS (
SELECT substring('strtrunc truncates strings longer than 60 characters, leaving the prefix and suffix visible', 1, s.num) AS value
FROM table(sequence(start=>40, stop=>80, step=>5)) AS s(num)
)
SELECT
data.value
, strtrunc(data.value) AS truncated
FROM data
ORDER BY data.value;
```
The preceding query produces the following output with all variants of the
routine:
```
value | truncated
----------------------------------------------------------------------------------+---------------------------------------------------------------
strtrunc truncates strings longer than 6 | strtrunc truncates strings longer than 6
strtrunc truncates strings longer than 60 cha | strtrunc truncates strings longer than 60 cha
strtrunc truncates strings longer than 60 characte | strtrunc truncates strings longer than 60 characte
strtrunc truncates strings longer than 60 characters, l | strtrunc truncates strings longer than 60 characters, l
strtrunc truncates strings longer than 60 characters, leavin | strtrunc truncates strings longer than 60 characters, leavin
strtrunc truncates strings longer than 60 characters, leaving the | strtrunc truncates strings lon ... 60 characters, leaving the
strtrunc truncates strings longer than 60 characters, leaving the pref | strtrunc truncates strings lon ... aracters, leaving the pref
strtrunc truncates strings longer than 60 characters, leaving the prefix an | strtrunc truncates strings lon ... ers, leaving the prefix an
strtrunc truncates strings longer than 60 characters, leaving the prefix and suf | strtrunc truncates strings lon ... leaving the prefix and suf
```
A possible improvement is to introduce parameters for the total length.
## Formatting bytes
Trino includes a built-in `format_number()` function. However it is using units
that don't work well with bytes. The following `format_data_size` routine can
format large values of bytes into a human readable string.
```sql theme={null}
FUNCTION format_data_size(input BIGINT)
RETURNS VARCHAR
BEGIN
DECLARE value DOUBLE DEFAULT CAST(input AS DOUBLE);
DECLARE result BIGINT;
DECLARE base INT DEFAULT 1024;
DECLARE unit VARCHAR DEFAULT 'B';
DECLARE format VARCHAR;
IF abs(value) >= base THEN
SET value = value / base;
SET unit = 'kB';
END IF;
IF abs(value) >= base THEN
SET value = value / base;
SET unit = 'MB';
END IF;
IF abs(value) >= base THEN
SET value = value / base;
SET unit = 'GB';
END IF;
IF abs(value) >= base THEN
SET value = value / base;
SET unit = 'TB';
END IF;
IF abs(value) >= base THEN
SET value = value / base;
SET unit = 'PB';
END IF;
IF abs(value) >= base THEN
SET value = value / base;
SET unit = 'EB';
END IF;
IF abs(value) >= base THEN
SET value = value / base;
SET unit = 'ZB';
END IF;
IF abs(value) >= base THEN
SET value = value / base;
SET unit = 'YB';
END IF;
IF abs(value) < 10 THEN
SET format = '%.2f';
ELSEIF abs(value) < 100 THEN
SET format = '%.1f';
ELSE
SET format = '%.0f';
END IF;
RETURN format(format, value) || unit;
END;
```
Below is a query to show how it formats a wide range of values.
```sql theme={null}
WITH
data AS (
SELECT CAST(pow(10, s.p) AS BIGINT) AS num
FROM table(sequence(start=>1, stop=>18)) AS s(p)
UNION ALL
SELECT -CAST(pow(10, s.p) AS BIGINT) AS num
FROM table(sequence(start=>1, stop=>18)) AS s(p)
)
SELECT
data.num
, format_data_size(data.num) AS formatted
FROM data
ORDER BY data.num;
```
The preceding query produces the following output:
```
num | formatted
----------------------+-----------
-1000000000000000000 | -888PB
-100000000000000000 | -88.8PB
-10000000000000000 | -8.88PB
-1000000000000000 | -909TB
-100000000000000 | -90.9TB
-10000000000000 | -9.09TB
-1000000000000 | -931GB
-100000000000 | -93.1GB
-10000000000 | -9.31GB
-1000000000 | -954MB
-100000000 | -95.4MB
-10000000 | -9.54MB
-1000000 | -977kB
-100000 | -97.7kB
-10000 | -9.77kB
-1000 | -1000B
-100 | -100B
-10 | -10.0B
0 | 0.00B
10 | 10.0B
100 | 100B
1000 | 1000B
10000 | 9.77kB
100000 | 97.7kB
1000000 | 977kB
10000000 | 9.54MB
100000000 | 95.4MB
1000000000 | 954MB
10000000000 | 9.31GB
100000000000 | 93.1GB
1000000000000 | 931GB
10000000000000 | 9.09TB
100000000000000 | 90.9TB
1000000000000000 | 909TB
10000000000000000 | 8.88PB
100000000000000000 | 88.8PB
1000000000000000000 | 888PB
```
## Charts
Trino already has a built-in `bar()` [color function](/sql/functions/color), but
it's using ANSI escape codes to output colors, and thus is only usable for
displaying results in a terminal. The following example shows a similar routine,
that only uses ASCII characters.
```sql theme={null}
FUNCTION ascii_bar(value DOUBLE)
RETURNS VARCHAR
BEGIN
DECLARE max_width DOUBLE DEFAULT 40.0;
RETURN array_join(
repeat('█',
greatest(0, CAST(floor(max_width * value) AS integer) - 1)), '')
|| ARRAY[' ', '▏', '▎', '▍', '▌', '▋', '▊', '▉', '█'][cast((value % (cast(1 as double) / max_width)) * max_width * 8 + 1 as int)];
END;
```
It can be used to visualize a value.
```sql theme={null}
WITH
data AS (
SELECT
cast(s.num as double) / 100.0 AS x,
sin(cast(s.num as double) / 100.0) AS y
FROM table(sequence(start=>0, stop=>314, step=>10)) AS s(num)
)
SELECT
data.x,
round(data.y, 4) AS y,
ascii_bar(data.y) AS chart
FROM data
ORDER BY data.x;
```
The preceding query produces the following output:
```text theme={null}
x | y | chart
-----+--------+-----------------------------------------
0.0 | 0.0 |
0.1 | 0.0998 | ███
0.2 | 0.1987 | ███████
0.3 | 0.2955 | ██████████▉
0.4 | 0.3894 | ██████████████▋
0.5 | 0.4794 | ██████████████████▏
0.6 | 0.5646 | █████████████████████▋
0.7 | 0.6442 | ████████████████████████▊
0.8 | 0.7174 | ███████████████████████████▊
0.9 | 0.7833 | ██████████████████████████████▍
1.0 | 0.8415 | ████████████████████████████████▋
1.1 | 0.8912 | ██████████████████████████████████▋
1.2 | 0.932 | ████████████████████████████████████▎
1.3 | 0.9636 | █████████████████████████████████████▌
1.4 | 0.9854 | ██████████████████████████████████████▍
1.5 | 0.9975 | ██████████████████████████████████████▉
1.6 | 0.9996 | ███████████████████████████████████████
1.7 | 0.9917 | ██████████████████████████████████████▋
1.8 | 0.9738 | ██████████████████████████████████████
1.9 | 0.9463 | ████████████████████████████████████▉
2.0 | 0.9093 | ███████████████████████████████████▍
2.1 | 0.8632 | █████████████████████████████████▌
2.2 | 0.8085 | ███████████████████████████████▍
2.3 | 0.7457 | ████████████████████████████▉
2.4 | 0.6755 | ██████████████████████████
2.5 | 0.5985 | ███████████████████████
2.6 | 0.5155 | ███████████████████▋
2.7 | 0.4274 | ████████████████▏
2.8 | 0.335 | ████████████▍
2.9 | 0.2392 | ████████▋
3.0 | 0.1411 | ████▋
3.1 | 0.0416 | ▋
```
It's also possible to draw more compacted charts. Following is a routine drawing
vertical bars:
```sql theme={null}
FUNCTION vertical_bar(value DOUBLE)
RETURNS VARCHAR
RETURN ARRAY[' ', '▁', '▂', '▃', '▄', '▅', '▆', '▇', '█'][cast(value * 8 + 1 as int)];
```
It can be used to draw a distribution of values, in a single column.
```sql theme={null}
WITH
measurements(sensor_id, recorded_at, value) AS (
VALUES
('A', date '2023-01-01', 5.0)
, ('A', date '2023-01-03', 7.0)
, ('A', date '2023-01-04', 15.0)
, ('A', date '2023-01-05', 14.0)
, ('A', date '2023-01-08', 10.0)
, ('A', date '2023-01-09', 1.0)
, ('A', date '2023-01-10', 7.0)
, ('A', date '2023-01-11', 8.0)
, ('B', date '2023-01-03', 2.0)
, ('B', date '2023-01-04', 3.0)
, ('B', date '2023-01-05', 2.5)
, ('B', date '2023-01-07', 2.75)
, ('B', date '2023-01-09', 4.0)
, ('B', date '2023-01-10', 1.5)
, ('B', date '2023-01-11', 1.0)
),
days AS (
SELECT date_add('day', s.num, date '2023-01-01') AS day
-- table function arguments need to be constant but range could be calculated
-- using: SELECT date_diff('day', max(recorded_at), min(recorded_at)) FROM measurements
FROM table(sequence(start=>0, stop=>10)) AS s(num)
),
sensors(id) AS (VALUES ('A'), ('B')),
normalized AS (
SELECT
sensors.id AS sensor_id,
days.day,
value,
value / max(value) OVER (PARTITION BY sensor_id) AS normalized
FROM days
CROSS JOIN sensors
LEFT JOIN measurements m ON day = recorded_at AND m.sensor_id = sensors.id
)
SELECT
sensor_id,
min(day) AS start,
max(day) AS stop,
count(value) AS num_values,
min(value) AS min_value,
max(value) AS max_value,
avg(value) AS avg_value,
array_join(array_agg(coalesce(vertical_bar(normalized), ' ') ORDER BY day), '') AS distribution
FROM normalized
WHERE sensor_id IS NOT NULL
GROUP BY sensor_id
ORDER BY sensor_id;
```
The preceding query produces the following output:
```text theme={null}
sensor_id | start | stop | num_values | min_value | max_value | avg_value | distribution
-----------+------------+------------+------------+-----------+-----------+-----------+--------------
A | 2023-01-01 | 2023-01-11 | 8 | 1.00 | 15.00 | 8.38 | ▃ ▄█▇ ▅▁▄▄
B | 2023-01-01 | 2023-01-11 | 7 | 1.00 | 4.00 | 2.39 | ▄▆▅ ▆ █▃▂
```
## Top-N
Trino already has a built-in [aggregate function](/sql/functions/aggregate) called
`approx_most_frequent()`, that can calculate most frequently occurring values.
It returns a map with values as keys and number of occurrences as values. Maps
are not ordered, so when displayed, the entries can change places on subsequent
runs of the same query, and readers must still compare all frequencies to find
the one most frequent value. The following is a routine returns ordered results
as a string.
```sql theme={null}
FUNCTION format_topn(input map)
RETURNS VARCHAR
NOT DETERMINISTIC
BEGIN
DECLARE freq_separator VARCHAR DEFAULT '=';
DECLARE entry_separator VARCHAR DEFAULT ', ';
RETURN array_join(transform(
reverse(array_sort(transform(
transform(
map_entries(input),
r -> cast(r AS row(key varchar, value bigint))
),
r -> cast(row(r.value, r.key) AS row(value bigint, key varchar)))
)),
r -> r.key || freq_separator || cast(r.value as varchar)),
entry_separator);
END;
```
Following is an example query to count generated strings:
```sql theme={null}
WITH
data AS (
SELECT lpad('', 3, chr(65+(s.num / 3))) AS value
FROM table(sequence(start=>1, stop=>10)) AS s(num)
),
aggregated AS (
SELECT
array_agg(data.value ORDER BY data.value) AS all_values,
approx_most_frequent(3, data.value, 1000) AS top3
FROM data
)
SELECT
a.all_values,
a.top3,
format_topn(a.top3) AS top3_formatted
FROM aggregated a;
```
The preceding query produces the following result:
```text theme={null}
all_values | top3 | top3_formatted
----------------------------------------------------+-----------------------+---------------------
[AAA, AAA, BBB, BBB, BBB, CCC, CCC, CCC, DDD, DDD] | {AAA=2, CCC=3, BBB=3} | CCC=3, BBB=3, AAA=2
```
# FUNCTION
Source: https://docs.upsolve.ai/sql/routines/function
## Synopsis
```text theme={null}
FUNCTION name ( [ parameter_name data_type [, ...] ] )
RETURNS type
[ LANGUAGE language]
[ NOT? DETERMINISTIC ]
[ RETURNS NULL ON NULL INPUT ]
[ CALLED ON NULL INPUT ]
[ SECURITY { DEFINER | INVOKER } ]
[ COMMENT description]
statements
```
## Description
Declare a SQL routine.
The `name` of the routine. [Inline routines](/sql/routines/introduction#inline-routines) can use a simple
string. [Catalog routines](/sql/routines/introduction#catalog-routines) must qualify the name of the catalog
and schema.
The list of parameters is a comma-separated list of names `parameter_name` and
data types `data_type`, see [data type](/sql/reference/types). An empty list, specified as
`()` is also valid.
The `type` value after the `RETURNS` keyword identifies the [data
type](/sql/reference/types) of the routine output.
The optional `LANGUAGE` characteristic identifies the language used for the
routine definition with `language`. Only `SQL` is supported.
The optional `DETERMINISTIC` or `NOT DETERMINISTIC` characteristic declares that
the routine is deterministic. This means that repeated routine calls with
identical input parameters yield the same result. For SQL language routines, a
routine is non-deterministic if it calls any non-deterministic routines and
functions. By default, routines are assume to have a deterministic
behavior.
The optional `RETURNS NULL ON NULL INPUT` characteristic declares that the
routine returns a `NULL` value when any of the input parameters are `NULL`.
The routine is not invoked with a `NULL` input value.
The `CALLED ON NULL INPUT` characteristic declares that the routine is invoked
with `NULL` input parameter values.
The `RETURNS NULL ON NULL INPUT` and `CALLED ON NULL INPUT` characteristics are
mutually exclusive, with `CALLED ON NULL INPUT` as the default.
The security declaration of `SECURITY INVOKER` or `SECURITY DEFINER` is only
valid for catalog routines. It sets the mode for processing the routine with the
permissions of the user who calls the routine (`INVOKER`) or the user who
created the routine (`DEFINER`).
The body of the routine can either be a simple single `RETURN` statement with an
expression, or compound list of `statements` in a `BEGIN` block. Routines must
contain a `RETURN` statement at the end of the top-level block, even if it's
unreachable.
## Examples
A simple catalog function:
```sql theme={null}
CREATE FUNCTION example.default.meaning_of_life()
RETURNS BIGINT
RETURN 42;
```
And used:
```sql theme={null}
SELECT example.default.meaning_of_life(); -- returns 42
```
Equivalent usage with an inline function:
```sql theme={null}
WITH FUNCTION meaning_of_life()
RETURNS BIGINT
RETURN 42
SELECT meaning_of_life();
```
Further examples of varying complexity that cover usage of the `FUNCTION`
statement in combination with other statements are available in the [SQL
routines examples documentation](/sql/routines/examples).
## See also
* [routines](/sql/routines/introduction)
* [begin](/sql/routines/begin)
* [return](/sql/routines/return)
# IF
Source: https://docs.upsolve.ai/sql/routines/if
## Synopsis
```text theme={null}
IF condition
THEN statements
[ ELSEIF condition THEN statements ]
[ ... ]
[ ELSE statements ]
END IF
```
## Description
The `IF THEN` statement is an optional construct to allow conditional processing
in [SQL routines](/sql/routines/introduction). Each `condition` following an `IF`
or `ELSEIF` must evaluate to a boolean. The result of processing the expression
must result in a boolean `true` value to process the `statements` in the `THEN`
block. A result of `false` results in skipping the `THEN` block and moving to
evaluate the next `ELSEIF` and `ELSE` blocks in order.
The `ELSEIF` and `ELSE` segments are optional.
## Examples
```sql theme={null}
FUNCTION simple_if(a bigint)
RETURNS varchar
BEGIN
IF a = 0 THEN
RETURN 'zero';
ELSEIF a = 1 THEN
RETURN 'one';
ELSE
RETURN 'more than one or negative';
END IF;
END
```
Further examples of varying complexity that cover usage of the `IF` statement in
combination with other statements are available in the [SQL routines examples
documentation](/sql/routines/examples).
## See also
* [routines](/sql/routines/introduction)
* [Conditional expressions using `IF`](/sql/functions/conditional)
# Introduction to SQL routines
Source: https://docs.upsolve.ai/sql/routines/introduction
A SQL routine is a custom, user-defined function authored by a user of Trino and
written in the SQL routine language. You can declare the routine body within a
[function](/sql/routines/function) block as [inline routines](#inline-routines) or [catalog
routines](#catalog-routines).
## Inline routines
An inline routine declares and uses the routine within a query processing
context. The routine is declared in a `WITH` block before the query:
```sql theme={null}
WITH
FUNCTION abc(x integer)
RETURNS integer
RETURN x * 2
SELECT abc(21);
```
Inline routine names must follow SQL identifier naming conventions, and cannot
contain `.` characters.
The routine declaration is only valid within the context of the query. A
separate later invocation of the routine is not possible. If this is desired,
use a [catalog routine](#catalog-routines).
Multiple inline routine declarations are comma-separated, and can include
routines calling each other, as long as a called routine is declared before
the first invocation.
```sql theme={null}
WITH
FUNCTION abc(x integer)
RETURNS integer
RETURN x * 2,
FUNCTION xyz(x integer)
RETURNS integer
RETURN abc(x) + 1
SELECT xyz(21);
```
Note that inline routines can mask and override the meaning of a built-in function:
```sql theme={null}
WITH
FUNCTION abs(x integer)
RETURNS integer
RETURN x * 2
SELECT abs(-10); -- -20, not 10!
```
## Catalog routines
Catalog routines must use a name that combines the catalog name and schema name
with the routine name, such as `example.default.power` for the `power` routine
in the `default` schema of the `example` catalog.
Invocation must use the fully qualified name, such as `example.default.power`.
## Routine declaration
Refer to the documentation for the [function](/sql/routines/function) keyword for more
details about declaring the routine overall. The routine body is composed with
statements from the following list:
* [begin](/sql/routines/begin)
* [case](/sql/routines/case)
* [declare](/sql/routines/declare)
* [if](/sql/routines/if)
* [iterate](/sql/routines/iterate)
* [leave](/sql/routines/leave)
* [loop](/sql/routines/loop)
* [repeat](/sql/routines/repeat)
* [return](/sql/routines/return)
* [set](/sql/routines/set)
* [while](/sql/routines/while)
Statements can also use [built-in functions and operators](/sql/functions/list) as well
as other routines, although recursion is not supported for routines.
Find simple examples in each statement documentation, and refer to the [example
documentation](/sql/routines/examples) for more complex use cases that combine
multiple statements.
## Labels
Routines can contain labels as markers for a specific block in the declaration
before the following keywords:
* `CASE`
* `IF`
* `LOOP`
* `REPEAT`
* `WHILE`
The label is used to name the block to continue processing with the `ITERATE`
statement or exit the block with the `LEAVE` statement. This flow control is
supported for nested blocks, allowing to continue or exit an outer block, not
just the innermost block. For example, the following snippet uses the label
`top` to name the complete block from `REPEAT` to `END REPEAT`:
```sql theme={null}
top: REPEAT
SET a = a + 1;
IF a <= 3 THEN
ITERATE top;
END IF;
SET b = b + 1;
UNTIL a >= 10
END REPEAT;
```
Labels can be used with the `ITERATE` and `LEAVE` statements to continue
processing the block or leave the block. This flow control is also supported for
nested blocks and labels.
## Recommendations
Processing routines can potentially be resource intensive on the cluster in
terms of memory and processing. Take the following considerations into account
when writing and running SQL routines:
* Some checks for the runtime behavior of routines are in place. For example,
routines that take longer to process than a hardcoded threshold are
automatically terminated.
* Avoid creation of arrays in a looping construct. Each iteration creates a
separate new array with all items and copies the data for each modification,
leaving the prior array in memory for automated clean up later. Use a [lambda
expression](/sql/functions/lambda) instead of the loop.
* Avoid concatenating strings in a looping construct. Each iteration creates a
separate new string and copying the old string for each modification, leaving
the prior string in memory for automated clean up later. Use a [lambda
expression](/sql/functions/lambda) instead of the loop.
* Most routines should declare the `RETURNS NULL ON NULL INPUT` characteristics
unless the code has some special handling for null values. You must declare
this explicitly since `CALLED ON NULL INPUT` is the default characteristic.
## Limitations
The following limitations apply to SQL routines.
* Routines must be declared before they are referenced.
* Recursion cannot be declared or processed.
* Mutual recursion can not be declared or processed.
* Queries cannot be processed in a routine.
Specifically this means that routines can not use `SELECT` queries to retrieve
data or any other queries to process data within the routine. Instead queries
can use routines to process data. Routines only work on data provided as input
values and only provide output data from the `RETURN` statement.
# ITERATE
Source: https://docs.upsolve.ai/sql/routines/iterate
## Synopsis
```text theme={null}
ITERATE label
```
## Description
The `ITERATE` statement allows processing of blocks in [SQL
routines](/sql/routines/introduction) to move processing back to the start of a
context block. Contexts are defined by a [`label`](/sql/routines/introduction#labels). If no label
is found, the functions fails with an error message.
## Examples
```sql theme={null}
FUNCTION count()
RETURNS bigint
BEGIN
DECLARE a int DEFAULT 0;
DECLARE b int DEFAULT 0;
top: REPEAT
SET a = a + 1;
IF a <= 3 THEN
ITERATE top;
END IF;
SET b = b + 1;
RETURN b;
END
```
Further examples of varying complexity that cover usage of the `ITERATE`
statement in combination with other statements are available in the [SQL
routines examples documentation](/sql/routines/examples).
## See also
* [routines](/sql/routines/introduction)
* [leave](/sql/routines/leave)
# LEAVE
Source: https://docs.upsolve.ai/sql/routines/leave
## Synopsis
```text theme={null}
LEAVE label
```
## Description
The `LEAVE` statement allows processing of blocks in [SQL
routines](/sql/routines/introduction) to move out of a specified context. Contexts
are defined by a [`label`](/sql/routines/introduction#labels). If no label is found, the functions
fails with an error message.
## Examples
The following function includes a `LOOP` labelled `top`. The conditional `IF`
statement inside the loop can cause the exit from processing the loop when the
value for the parameter `p` is 1 or less. This can be the case if the value is
passed in as 1 or less or after a number of iterations through the loop.
```sql theme={null}
FUNCTION my_pow(n int, p int)
RETURNS int
BEGIN
DECLARE r int DEFAULT n;
top: LOOP
IF p <= 1 THEN
LEAVE top;
END IF;
SET r = r * n;
SET p = p - 1;
END LOOP;
RETURN r;
END
```
Further examples of varying complexity that cover usage of the `LEAVE` statement
in combination with other statements are available in the [SQL routines examples
documentation](/sql/routines/examples).
## See also
* [routines](/sql/routines/introduction)
* [iterate](/sql/routines/iterate)
# LOOP
Source: https://docs.upsolve.ai/sql/routines/loop
## Synopsis
```text theme={null}
[label :] LOOP
statements
END LOOP
```
## Description
The `LOOP` statement is an optional construct in [SQL
routines](/sql/routines/introduction) to allow processing of a block of statements
repeatedly.
The block of `statements` is processed until an explicit use of `LEAVE` causes
processing to exit the loop. If processing reaches `END LOOP`, another iteration
of processing from the beginning starts. `LEAVE` statements are typically
wrapped in an `IF` statement that declares a condition to stop the loop.
The optional `label` before the `LOOP` keyword can be used to [name the
block](/sql/routines/introduction#labels).
## Examples
The following function counts up to `100` in a loop starting from the input
value `i` and returns the number of incremental steps in the loop to get to
`100`.
```sql theme={null}
FUNCTION to_one_hundred(i int)
RETURNS int
BEGIN
DECLARE count int DEFAULT 0;
abc: LOOP
IF i >= 100 THEN
LEAVE abc;
END IF
SET count = count + 1;
SET i = i + 1;
END LOOP;
RETURN;
END
```
Further examples of varying complexity that cover usage of the `LOOP` statement
in combination with other statements are available in the [SQL routines examples
documentation](/sql/routines/examples).
## See also
* [routines](/sql/routines/introduction)
* [leave](/sql/routines/leave)
# REPEAT
Source: https://docs.upsolve.ai/sql/routines/repeat
## Synopsis
```text theme={null}
[label :] REPEAT
statements
UNTIL condition
END REPEAT
```
## Description
The `REPEAT UNTIL` statement is an optional construct in [SQL
routines](/sql/routines/introduction) to allow processing of a block of statements
as long as a condition is met. The condition is validated as a last step of each
iteration.
The block of statements is processed at least once. After the first, and every
subsequent processing the expression `condidtion` is validated. If the result is
`true`, processing moves to `END REPEAT` and continues with the next statement in
the function. If the result is `false`, the statements are processed again.
The optional `label` before the `REPEAT` keyword can be used to [name the
block](/sql/routines/introduction#labels).
Note that a `WHILE` statement is very similar, with the difference that for
`REPEAT` the statements are processed at least once, and for `WHILE` blocks the
statements might not be processed at all.
## Examples
The following routine shows a routine with a `REPEAT` statement that runs until
the value of `a` is greater or equal to `10`.
```sql theme={null}
FUNCTION test_repeat(a bigint)
RETURNS bigint
BEGIN
REPEAT
SET a = a + 1;
UNTIL a >= 10
END REPEAT;
RETURN a;
END
```
Since `a` is also the input value and it is increased before the check the
routine always returns `10` for input values of `9` or less, and the input value
* 1 for all higher values.
Following are a couple of example invocations with result and explanation:
```sql theme={null}
SELECT test_repeat(5); -- 10
SELECT test_repeat(9); -- 10
SELECT test_repeat(10); -- 11
SELECT test_repeat(11); -- 12
SELECT test_repeat(12); -- 13
```
Further examples of varying complexity that cover usage of the `REPEAT`
statement in combination with other statements are available in the [SQL
routines examples documentation](/sql/routines/examples).
## See also
* [routines](/sql/routines/introduction)
* [loop](/sql/routines/loop)
* [while](/sql/routines/while)
# RETURN
Source: https://docs.upsolve.ai/sql/routines/return
## Synopsis
```text theme={null}
RETURN expression
```
## Description
Provide the value from a [SQL routines](/sql/routines/introduction) to the caller.
The value is the result of evaluating the expression. It can be a static value,
a declared variable or a more complex expression.
## Examples
The following examples return a static value, the result of an expression, and
the value of the variable x:
```sql theme={null}
RETURN 42;
RETURN 6 * 7;
RETURN x;
```
Further examples of varying complexity that cover usage of the `RETURN`
statement in combination with other statements are available in the [SQL
routines examples documentation](/sql/routines/examples).
All routines must contain a `RETURN` statement at the end of the top-level
block in the `FUNCTION` declaration, even if it's unreachable.
## See also
* [routines](/sql/routines/introduction)
* [function](/sql/routines/function)
# SET
Source: https://docs.upsolve.ai/sql/routines/set
## Synopsis
```text theme={null}
SET identifier = expression
```
## Description
Use the `SET` statement in [SQL routines](/sql/routines/introduction) to assign a
value to a variable, referenced by comma-separated `identifier`s. The
value is determined by evaluating the `expression` after the `=` sign.
Before the assignment the variable must be defined with a `DECLARE` statement.
The data type of the variable must be identical to the data type of evaluating
the `expression`.
## Examples
The following functions returns the value `1` after setting the counter variable
multiple times to different values:
```sql theme={null}
FUNCTION one()
RETURNS bigint
BEGIN
DECLARE counter tinyint DEFAULT 1;
SET counter = 0;
SET counter = counter + 2;
SET counter = counter / counter;
RETURN counter;
END
```
Further examples of varying complexity that cover usage of the `SET` statement
in combination with other statements are available in the [SQL routines examples
documentation](/sql/routines/examples).
## See also
* [routines](/sql/routines/introduction)
* [declare](/sql/routines/declare)
# WHILE
Source: https://docs.upsolve.ai/sql/routines/while
## Synopsis
```text theme={null}
[label :] WHILE condition DO
statements
END WHILE
```
## Description
The `WHILE` statement is an optional construct in [SQL
routines](/sql/routines/introduction) to allow processing of a block of statements
as long as a condition is met. The condition is validated as a first step of
each iteration.
The expression that defines the `condition` is evaluated at least once. If the
result is `true`, processing moves to `DO`, through following `statements` and
back to `WHILE` and the `condition`. If the result is `false`, processing moves
to `END WHILE` and continues with the next statement in the function.
The optional `label` before the `WHILE` keyword can be used to [name the
block](/sql/routines/introduction#labels).
Note that a `WHILE` statement is very similar, with the difference that for
`REPEAT` the statements are processed at least once, and for `WHILE` blocks the
statements might not be processed at all.
## Examples
```sql theme={null}
WHILE p > 1 DO
SET r = r * n;
SET p = p - 1;
END WHILE;
```
Further examples of varying complexity that cover usage of the `WHILE` statement
in combination with other statements are available in the [SQL routines examples
documentation](/sql/routines/examples).
## See also
* [routines](/sql/routines/introduction)
* [loop](/sql/routines/loop)
* [repeat](/sql/routines/repeat)