AI Functions can process up to 10,000 rows at a time. If you try to run one on a larger dataset, you’ll see a message saying that only 10,000 rows can be processed.
To work with more than 10,000 rows, split the data into branches, run the AI Function on each 10,000-row chunk, and then merge the results back together. Here’s an example with 30,000 rows.
-
Create branches
Click the “Create Branch Data Frame” button on any step, and create as many branches as you need (three for 30,000 rows). -
Narrow down the row range in each branch
From the plus button in the step area, select “Slice (Select Rows),” then enter row numbers in “Start Row” and “End Row.”- Branch 1:
1to10000 - Branch 2:
10001to20000 - Branch 3:
20001to30000
- Branch 1:
-
Run the AI Function in each branch
From the header menu of the target column, select “Create AI Function” → “Custom,” enter your prompt, and run it. Repeat with the same prompt in the remaining branches. -
Combine the results
Open one of the branches, select “Merge (Add Rows / Union)” from the step menu, check the remaining branches, and run it. All rows will be combined into a single data frame.
Note that the “Split data and process in parallel” checkbox in the AI Function dialog supports up to 10,000 rows when it’s on, and up to 3,000 rows when it’s off. Turning it on does not raise the limit beyond 10,000 rows, so use the approach above for larger datasets.
Also, if you update the main data frame, Exploratory automatically recalculates each branch based on its dependencies.
For more details, see the following pages: