feat(readers/web-firecrawl): migrate to Firecrawl v2 SDK (run-llama#19773)

Author: ftonato

docs/docs/examples/data_connectors/WebPageDemo.ipynb

@@ -290,6 +290,16 @@
     "Using Firecrawl to gather an entire website"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a41579cc",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%pip install firecrawl-py"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,

llama-index-integrations/readers/llama-index-readers-web/llama_index/readers/web/firecrawl_web/README.md

@@ -6,25 +6,31 @@
 
 1. **Install Firecrawl Package**: Ensure the `firecrawl-py` package is installed to use the Firecrawl Web Loader. Install it via pip with the following command:
 
-   ```bash
-   pip install firecrawl-py
-   ```
+```bash
+pip install 'firecrawl-py>=4.3.3'
+```
 
 2. **API Key**: Secure an API key from [Firecrawl.dev](https://www.firecrawl.dev/) to access the Firecrawl services.
 
 ### Using Firecrawl Web Loader
 
-- **Initialization**: Initialize the FireCrawlWebReader by providing the API key, the desired mode of operation (`crawl`, `scrape`, `search`, or `extract`), and any optional parameters for the Firecrawl API.
+- **Initialization**: Initialize the `FireCrawlWebReader` by providing the API key, the desired mode of operation (`crawl`, `scrape`, `map`, `search`, or `extract`), and any optional parameters for the Firecrawl API.
 
-  ```python
-  from llama_index.readers.web.firecrawl_web.base import FireCrawlWebReader
+```python
+from llama_index.readers.web.firecrawl_web.base import FireCrawlWebReader
 
-  firecrawl_reader = FireCrawlWebReader(
-      api_key="your_api_key_here",
-      mode="crawl",  # or "scrape" or "search" or "extract"
-      params={"additional": "parameters"},
-  )
-  ```
+firecrawl_reader = FireCrawlWebReader(
+    api_key="your_api_key_here",
+    mode="crawl",  # or "scrape" or "map" or "search" or "extract"
+    # Common params for the underlying Firecrawl client
+    # e.g. formats for content types and crawl limits
+    params={
+        "formats": ["markdown", "html"],  # for scrape or crawl
+        "limit": 100,  # for crawl
+        # "scrape_options": {"formats": ["markdown", "html"]},  # alternative shape for crawl
+    },
+)
+```
 
 - **Loading Data**: To load data, use the `load_data` method with the URL you wish to process.
 
@@ -43,8 +49,13 @@ Here is an example demonstrating how to initialize the FireCrawlWebReader, load
 # Initialize the FireCrawlWebReader with your API key and desired mode
 firecrawl_reader = FireCrawlWebReader(
     api_key="your_api_key_here",  # Replace with your actual API key
-    mode="crawl",  # Choose between "crawl", "scrape", "search" and "extract"
-    params={"additional": "parameters"},  # Optional additional parameters
+    mode="crawl",  # Choose between "crawl", "scrape", "map", "search" and "extract"
+    params={
+        # Provide formats for the content you want to retrieve
+        "formats": ["markdown", "html"],
+        # Limit the number of pages to crawl
+        "limit": 50,
+    },
 )
 
 # Load documents from Paul Graham's essay URL