ai agent that can scrape webpages
Configured with Agent, OpenAI Chat Model, Tool Http Request. Review the connections and setup before adapting this template.
Node categories in this workflow
- AI & models
- Web / API
- Transform
- Trigger
Icons describe general node roles, inferred from node types. Inspect the parameters for the configured action.
Drag to pan; use + and − to zoom. Reconstructed from the exported node positions; nothing runs here. Dashed lines represent non-main connections. Full node names and notes appear below.
How it works
The exported workflow contains the following configured nodes. Their connections are shown above; inspect the JSON for the complete logic, expressions, and external actions.
| Node | Category | Type | Operation |
|---|---|---|---|
| OpenAI Chat Model | AI & models | OpenAI Chat Model | See node parameters |
| OpenAI Chat Model1 | AI & models | OpenAI Chat Model | See node parameters |
| Activity Tool | Web / API | Tool Http Request | See node parameters |
| Set ChatInput1 | Transform | Edit Fields | See node parameters |
| AI Agent1 | AI & models | Agent | See node parameters |
| Set ChatInput | Transform | Edit Fields | See node parameters |
| When clicking ‘Test workflow’ | Trigger | Manual Trigger | See node parameters |
| AI Agent | AI & models | Agent | See node parameters |
| Webscraper Tool | Web / API | Tool Http Request | POST |
Setup
- Download the JSON and import it into an isolated n8n instance. Leave the workflow inactive.
- Connect your own credentials through n8n’s credential store. Replace
REPLACE_WITH_*values with your resource identifiers. - Inspect API destinations, Code nodes, database queries, file paths, and outbound actions.
- Run with synthetic data and harmless destinations. Enable triggers only after checking the actual outputs.
Requirements
An n8n instance with the listed node types and compatible versions. Service-connected workflows may require accounts, credentials, or paid APIs.
Credential types declared in the source
httpHeaderAuthopenAiApi
Workflow notes
AI agent that can scrape webpages
AI agent that can scrape webpages
Remake of https://n8n.io/workflows/2006-ai-agent-that-can-scrape-webpages/
Changes:
- Replaces Execute Workflow Tool and Subworkflow
- Replaces Response Formatting
Allow your AI to call an API to fetch data
Allow your AI to call an API to fetch data
Remake of https://n8n.io/workflows/2094-allow-your-ai-to-call-an-api-to-fetch-data/
Changes:
- Replaces Execute Workflow Tool and Subworkflow
- Replaces Manual Query Params Definitions
- Replaces Response Formatting
Try It Out!
Try It Out!
The HTTP tool is drastically simplifies API-enabled AI agents cutting down the number of workflow nodes by as much as 10!
- Available since v1.47.0
- Recommended for single purpose APIs which don't require much post-fetch formatting.
- If you require a chain of API calls, you may need to implement a subworkflow instead.
Need Help?
Join the Discord or ask in the Forum!
Happy Hacking!
ai agent that can scrape webpages
ai agent that can scrape webpages
Purpose and structure
This folder contains the ai agent that can scrape webpages workflow. Its executable nodes use agent, lmChatOpenAi, manualTrigger, set, toolHttpRequest. The table and connection map below are derived from the JSON, not an inferred guarantee of behavior.
Setup
Import inactive. Configure your own credentials and replace REPLACE_WITH_* values. Review external destinations and Code nodes before running.
Entry points
When clicking ‘Test workflow’
Verification
Sanitized source candidate; not runtime-tested. This overview is generated from node metadata. Read the folder README for the connection map, credential types, and license.
Attribution
https://github.com/Zie619/n8n-workflows/blob/94007c1445d9258a7da116646b79473e7c7c3282/workflows/Manual/0540_Manual_Stickynote_Automation_Webhook.json MIT; preserve source notices.