[RFC] 139 - BDD-Based E2E Testing Framework with Playwright

[arvinxx]

🎯 Executive Summary

This RFC proposes implementing a Behavior-Driven Development (BDD) approach for E2E testing using Playwright and Cucumber. The framework enables writing tests in natural language (Gherkin), making them readable and maintainable by both technical and non-technical team members.

Key Benefits

✅ Natural Language Tests - Tests written in Gherkin are human-readable
✅ Living Documentation - Tests serve as always-up-to-date documentation
✅ High Reusability - Step definitions can be reused across scenarios
✅ Better Collaboration - Product, QA, and Engineering can all contribute
✅ Traceability - Direct mapping between requirements and test cases

---

🏗️ Architecture Overview

Gherkin Features (Natural Language)
           ↓
    Cucumber (Test Runner)
           ↓
  Step Definitions (TypeScript)
           ↓
   Playwright (Browser API)
           ↓
    LobeHub Web App
           ↓
    Allure Report (Optional)

---

📁 Proposed Project Structure

root/
├── e2e/
│   ├── features/              # Gherkin feature files
│   │   ├── discover/
│   │   ├── chat/
│   │   └── settings/
│   ├── steps/                 # Step definitions
│   │   ├── common/
│   │   └── discover/
│   ├── support/               # Support utilities
│   │   ├── world.ts
│   │   ├── fixtures/
│   │   ├── helpers/
│   │   └── page-objects/
│   ├── config/
│   └── reports/

---

📝 Example: Gherkin Feature

@discover @navigation
Feature: Discover Navigation
  As a user
  I want to navigate between different discover tabs
  So that I can explore assistants, models, providers, and MCP tools

  @DISCOVER-NAV-001 @P0 @smoke
  Scenario: Navigate between tabs
    When I click on "Assistants" tab
    Then I should be on "/discover/assistant"
    And the "Assistants" tab should be highlighted

---

🔧 Implementation Phases

Phase 1: Setup

• Install dependencies (@cucumber/cucumber, @playwright/test)
• Configure Cucumber and Playwright
• Set up project structure

Phase 2: Add Test IDs

• Add data-testid attributes to components
• Document test ID conventions

Phase 3: Write Tests

• Start with smoke tests
• Implement common step definitions
• Build feature-specific tests

Phase 4: Organize Data

• Create test fixtures
• Set up test data management

Phase 5: CI Integration

• Configure GitHub Actions
• Set up Allure reporting
• Deploy reports to GitHub Pages

---

🏷️ Tagging Strategy

Tests can be organized and filtered using tags:

@P0 @smoke @discover @desktop @DISCOVER-NAV-001

Run examples:

npm run test:e2e -- --tags "@smoke"           # Smoke tests only
npm run test:e2e -- --tags "@P0"              # Critical tests
npm run test:e2e -- --tags "@discover"        # Discover module
npm run test:e2e -- --tags "not @flaky"       # Exclude flaky tests

---

👥 Feedback Welcome!

Please share your thoughts, concerns, and suggestions below. We're particularly interested in:

• Architecture feedback
• Testing strategy recommendations
• Tool/framework suggestions
• Timeline and resource concerns