camel.loaders.scrapegraph_reader.md

ScrapeGraphAI

class ScrapeGraphAI:

ScrapeGraphAI allows you to perform AI-powered web scraping and searching.

Parameters:

• api_key (Optional[str]): API key for authenticating with the ScrapeGraphAI API.
• References:
• https: //scrapegraph.ai/

init

def __init__(self, api_key: Optional[str] = None):

search

def search(self, user_prompt: str):

Perform an AI-powered web search using ScrapeGraphAI.

Parameters:

• user_prompt (str): The search query or instructions.

Returns:

Dict[str, Any]: The search results including answer and reference URLs.

scrape

def scrape(
    self,
    website_url: str,
    user_prompt: str,
    website_html: Optional[str] = None
):

Perform AI-powered web scraping using ScrapeGraphAI.

Parameters:

• website_url (str): The URL to scrape.
• user_prompt (str): Instructions for what data to extract.
• website_html (Optional[str]): Optional HTML content to use instead of fetching from the URL.

Returns:

Dict[str, Any]: The scraped data including request ID and result.

close

def close(self):

Close the ScrapeGraphAI client connection.