Project documentation

View Shopping Agent documentation | View Chatbot documentation

---

Shopping Agent

Overview

ShoppingAgent is an intelligent order-analysis agent that uses browser automation, screenshot recognition, and large-model analysis to extract and summarize order information and generate Excel files, including data from back-office systems that cannot be exported directly. It uses FastAPI for the backend and Streamlit for the frontend to provide an automated agent workflow with a simple interface.

Features

• Natural-language interaction through a chat interface.
• Browser automation for opening pages and capturing screenshots.
• Full-screen, element-selection, and manual-region screenshots.
• Batch analysis and information extraction across multiple order screenshots.
• Export of analysis results to Excel.
• Configurable server, model, browser, and related parameters.

Quick start

Requirements

• Python 3.8+
• Google Chrome 140.0 or later; the latest stable release is recommended.
• Chrome driver, available under dist/resourece.

Installation

1. Clone the repository.
2. Install dependencies:

pip install -r requirements.txt

Run the application

1. Start the backend:

python -m app.main

2. In a new terminal, start the frontend:

streamlit run app/frontend.py

3. The browser should open automatically. You can also visit http://localhost:8501 manually.

Usage

1. Open the order page: enter the order-page URL and click the button to open it.
2. Capture order screenshots: select full-screen or manual-region capture. For a manual region, first capture the full page, drag over the required order area, and capture the selected region.
3. Analyze orders: run batch order analysis so the system processes all screenshots and extracts order information.
4. Review and export: inspect the results in the side panel and export them to Excel.

Natural-language control

The chat window on the left can be used to control the system with natural-language instructions.

System configuration

The system settings sidebar lets you configure:

• Server address and port.
• AI model settings and API keys.
• Chrome browser and driver paths.

Configuration changes require an application restart.

Directory structure

project-root/
├── app/
│   ├── __init__.py
│   ├── main.py          # Backend entry point
│   ├── frontend.py      # Frontend interface
│   ├── config.py        # Configuration
│   └── utils/           # Utility functions
├── screenshots/         # Captured screenshots
├── excels/              # Excel exports
├── .env                 # Environment variables
└── requirements.txt     # Dependencies

Troubleshooting

1. Browser will not start: verify CHROME_PATH, make sure Chrome is installed, update Chrome if necessary, and on Windows avoid installation paths containing non-ASCII or special characters.
2. Screenshot capture fails: wait for the page to load fully, try another capture mode, make sure the browser is not minimized, and verify that Chrome and chromedriver versions match.
3. Analysis is inaccurate: use clear screenshots, crop to the order information when possible, and verify the API key and network connection.
4. Frontend cannot connect to backend: make sure the backend is running, verify the configured server address, and check that the port is available.

Dependencies

Core

• python-dotenv>=1.0.0 — environment variables
• fastapi>=0.103.1 — backend API framework
• uvicorn>=0.23.2 — ASGI server
• streamlit>=1.27.0 — frontend framework
• requests>=2.31.0 — HTTP requests
• pandas>=2.1.1 — data processing
• openpyxl>=3.1.2 — Excel processing
• pillow>=10.0.1 — image processing

Browser automation

• selenium>=4.14.0 — browser automation compatible with Chrome 112+
• webdriver-manager>=4.0.1 — WebDriver management and automatic Chrome version matching

Image and multimodal processing

• python-multipart>=0.0.6 — multipart form data

---

Switch to Chatbot documentation

---

Chatbot

ChatGPT popularized GPT-style conversational systems, and this project is evolving in the same direction. It is a Chinese chatbot project that can be trained on custom corpora and is open for experimentation, discussion, stars, and forks.

Seq2Seq example results at 50% training progress

Roadmap

V1.1 — 2024-09-30

1. Add a MindSpore version and prioritize GPT models, RLHF, and related features there.
2. Split the engineering architecture into Seq2Seq and GPT branches while continuing to support multiple AI frameworks.

V1.2 — planned for 2024-12-30

1. Add MiniGPT-4-style multimodal text-and-image conversation capabilities.
2. Improve distributed-cluster training and RLHF-related capabilities.

Seq2Seq execution order

A Xiaohuangji corpus can be downloaded from: https://github.com/zhaoyingjun/chatbot/blob/master/chineseChatbotWeb-tf2.0/seq2seqChatbot/train_data/xiaohuangji50w_nofenci.conv

1. After downloading the code and corpus, place the corpus in train_data. Hyperparameters are configured in config/seq2seq.ini.
2. Run the data preprocessor (data_utls.py), then execute.py, then app.py for the visual conversation interface.
3. For large-scale distributed training, follow the Horovod launch pattern:

horovodrun -np n -H host1_ip:port,host2_ip:port,hostn_ip:port python3 excute.py

Recommended training environment

• Ubuntu 18.04
• Python 3.6

TensorFlow 2.x

• tensorflow 2.6.0
• flask 0.11.1
• horovod 0.24 for distributed training

PyTorch

• torch 1.11.0
• flask 0.11.1

Contact

QQ: 934389697

---

Switch to Shopping Agent documentation