Skip to content

Commit 5502a50

Browse files
committed
chore: add missing sections (#263)
### TL;DR Renamed "Configuring Retries" to "Retrying Steps" and added a new "Validation Steps" guide to improve documentation around error handling patterns. ### What changed? - Renamed `/build/configuring-retries/` to `/build/retrying-steps/` for better clarity - Added a new guide on `/build/validation-steps/` explaining how to properly handle validation errors - Enhanced the retry documentation with sections on understanding different failure types - Updated all internal links and navigation references to point to the renamed pages - Refined the LLM content exclusion list in `astro.config.mjs` to be more specific about which index pages to exclude ### How to test? 1. Build the website and verify the new page structure: - Confirm `/build/retrying-steps/` loads correctly with the enhanced content - Verify the new `/build/validation-steps/` page is accessible - Check that all links to the old "Configuring Retries" page redirect properly 2. Review the content of both pages to ensure they provide clear guidance on: - Distinguishing between transient and permanent failures - Setting up validation steps with `maxAttempts: 1` - Following best practices for synchronous validation ### Why make this change? Users were confused about when to use retries versus when to fail fast. The documentation needed clearer guidance on handling different types of errors appropriately. By separating validation patterns into their own guide and enhancing the retry documentation, we provide more specific guidance on error handling best practices, helping users build more robust flows that don't waste resources on unnecessary retries.
1 parent 46a440e commit 5502a50

File tree

7 files changed

+170
-17
lines changed

7 files changed

+170
-17
lines changed

pkgs/website/astro.config.mjs

Lines changed: 19 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -152,15 +152,22 @@ export default defineConfig({
152152
starlightLlmsTxt({
153153
exclude: [
154154
'index',
155-
'**/index',
155+
// Navigation-only index files (pure CardGrid hubs with no unique content)
156+
'build/index',
157+
'concepts/index',
158+
'deploy/index',
159+
'reference/index',
160+
'tutorials/index',
161+
'comparisons/index',
162+
// Tutorials (lengthy, patterns covered elsewhere)
156163
'tutorials/ai-web-scraper/*',
157-
'concepts/naming-steps',
158-
'deploy/tune-flow-config',
159-
'get-started/faq',
164+
// News/blog and non-technical content
160165
'news/**',
161166
'hire/**',
162-
'build/configuring-retries',
163-
'build/delaying-steps',
167+
'author/**',
168+
// Note: The following index files ARE included because they have valuable content:
169+
// - build/starting-flows/index (comparison guide for starting flows)
170+
// - reference/configuration/index (config philosophy and structure)
164171
],
165172
promote: [
166173
'get-started/installation',
@@ -225,8 +232,12 @@ export default defineConfig({
225232
link: '/build/organize-flow-code/',
226233
},
227234
{
228-
label: 'Configuring retries',
229-
link: '/build/configuring-retries/',
235+
label: 'Retrying steps',
236+
link: '/build/retrying-steps/',
237+
},
238+
{
239+
label: 'Validation steps',
240+
link: '/build/validation-steps/',
230241
},
231242
{
232243
label: 'Delaying steps',

pkgs/website/src/content/docs/build/delaying-steps.mdx

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ import { Aside, CardGrid, LinkCard } from "@astrojs/starlight/components";
1010
Use [`startDelay`](/reference/configuration/step-execution/#startdelay) to schedule steps for execution after a specified time period. Delays are relative to when the step's dependencies complete.
1111

1212
<Aside type="tip">
13-
For retry configuration patterns, see [Configuring Retries](/build/configuring-retries/).
13+
For retry configuration patterns, see [Retrying Steps](/build/retrying-steps/).
1414
</Aside>
1515

1616
For detailed information about each configuration option, see the [Step Execution Options](/reference/configuration/step-execution/) reference.
@@ -82,8 +82,8 @@ new Flow({
8282
description="Complete reference for all configuration options and their defaults"
8383
/>
8484
<LinkCard
85-
title="Configuring Retries"
86-
href="/build/configuring-retries/"
85+
title="Retrying Steps"
86+
href="/build/retrying-steps/"
8787
description="Configure retry policies for different reliability requirements"
8888
/>
8989
<LinkCard

pkgs/website/src/content/docs/build/index.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,8 @@ Now that you've created your first flow, learn how to structure your code, integ
1616
description="Structure your flows and tasks for maintainability and reusability"
1717
/>
1818
<LinkCard
19-
title="Configuring retries"
20-
href="/build/configuring-retries/"
19+
title="Retrying steps"
20+
href="/build/retrying-steps/"
2121
description="Configure retry policies for different step reliability requirements"
2222
/>
2323
<LinkCard

pkgs/website/src/content/docs/build/configuring-retries.mdx renamed to pkgs/website/src/content/docs/build/retrying-steps.mdx

Lines changed: 56 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: Configuring Retries
2+
title: Retrying Steps
33
description: Configure retry policies for different step reliability requirements.
44
sidebar:
55
order: 25
@@ -15,6 +15,56 @@ For scheduling delays between steps, see [Delaying Steps](/build/delaying-steps/
1515

1616
For detailed information about each configuration option, see the [Step Execution Options](/reference/configuration/step-execution/) reference.
1717

18+
## Understanding Failure Types
19+
20+
Not all failures should retry. Understanding the difference helps you configure appropriate retry policies.
21+
22+
### Transient Failures
23+
24+
Temporary problems that might succeed on retry:
25+
- Network timeouts
26+
- Rate limiting (429 responses)
27+
- Temporary service unavailability (503 responses)
28+
- Database connection issues
29+
30+
Configure with retries:
31+
```typescript
32+
.step({
33+
slug: 'fetchExternalData',
34+
maxAttempts: 5, // Retry transient failures
35+
baseDelay: 2,
36+
}, async (input) => await fetchFromAPI(input.run.url))
37+
```
38+
39+
### Permanent Failures
40+
41+
Problems that will never succeed on retry:
42+
- Invalid input format (malformed email, negative numbers)
43+
- Missing required fields
44+
- Business rule violations
45+
- Schema validation errors
46+
47+
Configure without retries:
48+
```typescript
49+
.step({
50+
slug: 'validInput',
51+
maxAttempts: 1, // No retries for validation
52+
}, (input) => {
53+
if (!input.run.email) throw new Error('Email required');
54+
return input.run;
55+
})
56+
```
57+
58+
<Aside type="note">
59+
`maxAttempts: 1` means "run once, do not retry". If the step fails, it fails immediately without retry attempts.
60+
</Aside>
61+
62+
<Aside type="caution" title="Current Limitation">
63+
pgflow does not distinguish between transient and permanent failures automatically. All exceptions trigger retry logic based on `maxAttempts`. Use `maxAttempts: 1` for steps that perform validation or other operations that should fail fast.
64+
</Aside>
65+
66+
For detailed guidance on validation patterns, see [Validation Steps](/build/validation-steps/).
67+
1868
## Guiding Principle
1969

2070
**Set conservative flow-level defaults, override per-step as needed.**
@@ -56,6 +106,11 @@ new Flow({
56106
## Learn More
57107

58108
<CardGrid>
109+
<LinkCard
110+
title="Validation Steps"
111+
href="/build/validation-steps/"
112+
description="Use explicit validation steps to fail fast on invalid input"
113+
/>
59114
<LinkCard
60115
title="Step Execution Options"
61116
href="/reference/configuration/step-execution/"
Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
---
2+
title: Validation Steps
3+
description: Use explicit validation steps to fail fast on invalid input
4+
sidebar:
5+
order: 30
6+
---
7+
8+
import { Aside } from '@astrojs/starlight/components';
9+
10+
Validation steps catch invalid input early, before expensive operations run. Structure them to fail fast with no retries.
11+
12+
## Why Explicit Validation Steps?
13+
14+
pgflow retries all exceptions based on `maxAttempts`. Without explicit validation, input errors waste retry attempts:
15+
16+
```typescript
17+
// Without validation - wastes retry attempts on invalid input
18+
new Flow<{ email: string }>({ slug: 'sendEmail', maxAttempts: 5 })
19+
.step(
20+
{ slug: 'send' },
21+
async (input) => {
22+
if (!input.run.email.includes('@')) {
23+
throw new Error('Invalid email'); // Retries 5 times!
24+
}
25+
return await sendEmail(input.run.email);
26+
}
27+
)
28+
```
29+
30+
With explicit validation, failures stop immediately:
31+
32+
```typescript
33+
// With validation - fails immediately on invalid input
34+
new Flow<{ email: string }>({ slug: 'sendEmail' })
35+
.step(
36+
{ slug: 'validInput', maxAttempts: 1 },
37+
(input) => {
38+
if (!input.run.email.includes('@')) {
39+
throw new Error('Invalid email');
40+
}
41+
return input.run;
42+
}
43+
)
44+
.step(
45+
{ slug: 'send', dependsOn: ['validInput'], maxAttempts: 5 },
46+
async (input) => await sendEmail(input.validInput.email)
47+
)
48+
```
49+
50+
## Keep Validation Synchronous
51+
52+
Validation steps should be fast, synchronous functions that check input format and structure. Avoid async operations like database queries or API calls - those belong in separate steps with appropriate retry configuration.
53+
54+
```typescript
55+
// Good: Fast, synchronous validation
56+
.step(
57+
{ slug: 'validOrder', maxAttempts: 1 },
58+
(input) => {
59+
const { amount, items } = input.run;
60+
61+
if (amount <= 0) throw new Error('amount must be positive');
62+
if (!items?.length) throw new Error('items cannot be empty');
63+
64+
return input.run;
65+
}
66+
)
67+
68+
// Bad: Async checks in validation
69+
.step(
70+
{ slug: 'validCustomer', maxAttempts: 1 },
71+
async (input) => {
72+
// Database lookups belong in separate steps with retries
73+
const exists = await checkCustomerExists(input.run.customerId);
74+
if (!exists) throw new Error('Customer not found');
75+
return input.run;
76+
}
77+
)
78+
```
79+
80+
<Aside type="tip">
81+
Validation steps check **format and structure** (email format, positive numbers, array length). Existence checks and external validations belong in regular steps with retry configuration.
82+
</Aside>
83+
84+
## See Also
85+
86+
- [Retrying Steps](/build/retrying-steps/) - Understanding failure types and retry policies
87+
- [Context Object](/concepts/context-object/) - Accessing validated input in dependent steps

pkgs/website/src/content/docs/index.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -372,7 +372,7 @@ npx pgflow@latest install
372372
</Card>
373373

374374
<Card title="🔄 Automatic retries">
375-
Built-in retry logic with exponential backoff for flaky AI APIs. When OpenAI times out or rate-limits, only that step retries - your workflow continues. Configure max attempts and delays per step, no retry code needed. [Learn more →](/build/configuring-retries/)
375+
Built-in retry logic with exponential backoff for flaky AI APIs. When OpenAI times out or rate-limits, only that step retries - your workflow continues. Configure max attempts and delays per step, no retry code needed. [Learn more →](/build/retrying-steps/)
376376
</Card>
377377

378378
<Card title="⚡ Parallel array processing">

pkgs/website/src/content/docs/reference/configuration/index.mdx

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,9 +44,9 @@ new Flow({ slug: 'my_flow' })
4444

4545
<CardGrid>
4646
<LinkCard
47-
title="Configuring Retries"
47+
title="Retrying Steps"
4848
description="Configure retry policies for different step reliability requirements."
49-
href="/build/configuring-retries/"
49+
href="/build/retrying-steps/"
5050
/>
5151
<LinkCard
5252
title="Delaying Steps"

0 commit comments

Comments
 (0)