Skip to content
UXClaim
User Research

Web Docs Capability Research Skill

Research website capabilities and compile help documentation into structured markdown files for analysis.

View source More in User Research

What it does

This skill automates the research of website and product capabilities by crawling help documentation and compiling it into structured markdown files. It extracts capability information, feature modules, and operational procedures scattered across multiple pages.

How it works

The skill crawls help documentation from a specified base URL and organizes content into standardized outputs including comprehensive help docs, capability reports with evidence links, and quality assessments. It supports bilingual documentation and custom content selectors.

Key outputs

  • help_docs_full.md: Complete consolidated help documentation
  • capability_report.md: Research report with referenced source pages
  • coverage.json: Documentation coverage metrics
  • qa_report.json: Page quality classification (404s, access restrictions, content issues)
  • run_summary.json: Execution summary and statistics

Use cases

Ideal for competitive research, product analysis, and documentation audits. Helps teams understand competitor capabilities, identify documentation gaps, and prepare content for knowledge platforms like NotebookLM.

Who benefits

Product managers conducting market research, UX researchers analyzing competitor offerings, content designers auditing documentation, and product strategists evaluating feature landscapes.

Frequently asked questions

How do I run the capability research skill?
Use the command: `python3 scripts/crawl_and_research.py --base-url https://docs.example.com --mode all --lang bilingual --content-selector "#content"`. Adjust the base-url and content-selector parameters for your target website.
What does capability research skill do?
It automatically crawls website help documentation, extracts product capabilities and features, and compiles them into structured markdown files with capability reports, quality assessments, and coverage metrics.
What output files does the skill generate?
Five main outputs: help_docs_full.md (consolidated docs), capability_report.md (with source evidence), coverage.json (metrics), qa_report.json (quality classification), and run_summary.json (execution summary).
Can I use this for competitor research?
Yes, it's designed for analyzing competitor documentation and capabilities. Point it at competitor help docs to extract and compare feature sets and operational procedures.
Does it support multiple languages?
Yes, the skill supports bilingual mode to handle documentation in multiple languages, useful for researching products with international documentation.
How do I customize the content extraction?
Use the --content-selector parameter to specify CSS selectors matching your target website's content area. Common examples: "#content", ".main-content", or "article".
What is the difference between help_docs_full.md and capability_report.md?
help_docs_full.md contains complete consolidated documentation. capability_report.md is a structured research report that highlights key capabilities with links to source pages for reference and evidence.
Can the skill identify broken links or inaccessible pages?
Yes, it classifies page quality in qa_report.json, identifying 404 errors, access restrictions, and pages with no actual content.

Glossary

Capability Report
A structured document that outlines product features and functionalities with references to source documentation pages, providing evidence for each capability identified.
Content Selector
A CSS selector string used to identify the main content area of web pages during crawling, allowing the skill to extract only relevant text and ignore navigation or sidebar elements.
Coverage Metrics
Statistical data about the scope of documented content, including number of pages crawled, content completeness, and documentation breadth across different feature areas.

More in User Research

All →
User Research

Ditto Product Research

Run customer research, pricing tests, and product validation using 300K+ AI personas with 92% focus group overlap.

Ask-Ditto
User Research

Reddit Intel

Convert Reddit discussions into bilingual Excel reports for competitor analysis, user insights, and market research.

carrielabs
User Research

Research Team Tutorial

Build a multi-agent AI research system using Claude Code that extracts data from papers, performs analysis, writes articles, and reviews its own work.

Tristan578
User Research

UX Discovery Interviewer

AI-powered agent skill that conducts structured UX research interviews, transforming vague product ideas into actionable discovery insights.