Skip to content

Commit c27d22b

Browse files
PRAteek-singHWYPRAteek-singHWY
committed
docs: add macOS local development setup guide
1 parent 701dac2 commit c27d22b

File tree

1 file changed

+228
-93
lines changed

1 file changed

+228
-93
lines changed

‎README.md‎

Lines changed: 228 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -1,129 +1,264 @@
1-
# OpenCRE readme
1+
# OpenCRE
22

3-
Go to https://www.opencre.org to see OpenCRE working and more explanation.
4-
OpenCRE stands for Open Common Requirement enumeration. It is an interactive content linking platform for uniting security standards and guidelines. It offers easy and robust access to relevant information when designing, developing, testing and procuring secure software.
3+
Go to [https://www.opencre.org](https://www.opencre.org) to see OpenCRE working and more explanation.
4+
5+
OpenCRE stands for **Open Common Requirement Enumeration**. It is an interactive content linking platform for uniting security standards and guidelines. It offers easy and robust access to relevant information when designing, developing, testing and procuring secure software.
6+
7+
---
8+
9+
## What is OpenCRE?
510

611
OpenCRE consists of:
7-
- The application: a python web and cli application to access the data, running publicly at opencre.org
8-
- The catalog data: a catalog of Common Requirements (CREs)
9-
- The mapping data: links from each CRE to relevant sections in a range of standards
10-
- Tools and guidelines to contribute to the data and to run the application locally
1112

12-
# Contribute code or mappings
13-
To see how you can contribute to the application or to the data (catalog or standard mappings), see [Contributing](docs/CONTRIBUTING.md).
13+
* **The application**: a Python web and CLI application to access the data, running publicly at opencre.org
14+
* **The catalog data**: a catalog of Common Requirements (CREs)
15+
* **The mapping data**: links from each CRE to relevant sections in a range of standards
16+
* **Tools and guidelines** to contribute to the data and to run the application locally
17+
18+
---
19+
20+
## Contribute Code or Mappings
21+
22+
To see how you can contribute to the application or to the data (catalog or standard mappings), see **Contributing**.
23+
1424
We really welcome you!
1525

16-
# Roadmap
17-
For a roadmap please see the [issues](https://github.com/OWASP/OpenCRE/issues).
26+
---
27+
28+
## Roadmap
29+
30+
For a roadmap please see the **Issues**.
31+
32+
---
33+
34+
## Running Your Own OpenCRE
35+
36+
You are free to use the public OpenCRE application at [https://www.opencre.org](https://www.opencre.org).
37+
You can also run your own instance if you want to include your own security standards and guidelines. We call that **myOpenCRE**.
38+
39+
---
40+
41+
## Running OpenCRE Locally
42+
43+
### Docker (Recommended)
44+
45+
The easiest way to run OpenCRE locally is by using Docker:
46+
47+
```bash
48+
docker run -p 5000:5000 ghcr.io/owasp/opencre/opencre:latest
49+
```
50+
51+
After the container has finished downloading the remote information, you can access the application at:
52+
53+
```
54+
http://localhost:5000
55+
```
56+
57+
If you want to develop OpenCRE or Docker is not available in your environment, you can run it via the command line.
58+
59+
---
60+
61+
## Command Line Setup
62+
63+
To run OpenCRE outside Docker you need:
64+
65+
* Python 3
66+
* Yarn
67+
* virtualenv
68+
69+
Clone the repository:
70+
71+
```bash
72+
git clone https://github.com/OWASP/OpenCRE.git
73+
cd OpenCRE
74+
```
75+
76+
(Recommended) Create and activate a Python virtual environment:
77+
78+
```bash
79+
python3 -m venv venv
80+
source venv/bin/activate
81+
```
82+
83+
Install dependencies:
84+
85+
```bash
86+
make install
87+
```
88+
89+
Download the latest CRE graph from upstream:
90+
91+
```bash
92+
make upstream-sync
93+
```
94+
95+
> ⚠️ Until Issue #534 is fixed, Gap Analysis results may not be available locally.
96+
97+
Run OpenCRE locally:
98+
99+
```bash
100+
make dev-flask
101+
```
102+
103+
---
104+
105+
## macOS Local Development Setup
106+
107+
This section documents common issues and solutions when setting up OpenCRE locally on **macOS**, based on contributor experience.
108+
109+
### Tested Environment
110+
111+
* macOS (Intel & Apple Silicon)
112+
* Python 3.10 – 3.13
113+
* Node.js 18+
114+
* Yarn 1.x
115+
116+
---
117+
118+
### 1. Virtual Environment Setup
119+
120+
We recommend using Python’s built-in `venv`:
121+
122+
```bash
123+
python3 -m venv venv
124+
source venv/bin/activate
125+
```
126+
127+
> **Important Note**
128+
> The Makefile currently invokes `virtualenv` internally.
129+
> On macOS, this can cause setup failures even when `venv` is used.
130+
>
131+
> To avoid this, install `virtualenv` inside the active virtual environment:
18132
19-
# Running your own OpenCRE
133+
```bash
134+
pip install virtualenv
135+
```
20136

21-
You are free to use the public opencre application at opencre.org. Apart from that, you can run your own if you want to include your own security standards and guidelines for example. We call that myOpenCRE.
137+
---
22138

23-
### Locally
139+
### 2. Installing Dependencies
24140

25-
#### Docker
26-
The easiest way to run OpenCRE locally is by running the published docker container.
27-
You can do so by running:
28-
`docker run -p 5000:5000 ghcr.io/owasp/opencre/opencre:latest`
29-
After the container has finished downloading the remote information you can access it in [localhost](http://127.0.0.1:5000)
30-
If you want to develop on OpenCRE or docker is not available in your environment, you can alternatively run it via CLI
141+
```bash
142+
make install
143+
```
31144

32-
#### Command Line
145+
If you encounter the error:
33146

34-
To run outside of Docker you need to install OpenCRE.
35-
To install this application you need python3, yarn and virtualenv.
36-
* Clone the repository:
37-
<pre>git clone https://github.com/OWASP/common-requirement-enumeration </pre>
147+
```text
148+
make: virtualenv: No such file or directory
149+
```
38150

39-
* (Recommend) Create and activate a Python virtual environment:
40-
<pre>python3 -m venv venv
41-
source venv/bin/activate </pre>
151+
Ensure that:
42152

43-
* Install dependencies
44-
<pre> make install </pre>
153+
* the virtual environment is activated
154+
* `virtualenv` is installed inside the environment
45155

46-
* Download the latest CRE graph from upstream by running
47-
<pre>make upstream-sync </pre>
48-
Keep in mind that until [Issue #534](https://github.com/OWASP/OpenCRE/issues/534) is fixed you won't have access to gap analysis results locally
156+
---
49157

50-
* To run CRE locally then you can do:
51-
<pre> make dev-flask </pre>
158+
### 3. Syncing Upstream Data (Known Limitation)
52159

53-
To run the CLI application, you can run
54-
<pre>python cre.py --help</pre>
160+
```bash
161+
make upstream-sync
162+
```
55163

56-
To download a remote cre spreadsheet locally you can run
57-
<pre>python cre.py --review --from_spreadsheet < google sheets url></pre>
164+
> **Known Issue**
165+
> On macOS, this command may fail with:
166+
>
167+
> ```text
168+
> sqlite3.OperationalError: no such table: cre
169+
> ```
170+
>
171+
> This is a known upstream limitation (see Issue #534).
172+
> The application can still be run locally for development even if this step fails.
58173
59-
To add a remote spreadsheet to your local database you can run
60-
<pre>python cre.py --add --from_spreadsheet < google sheets url></pre>
174+
---
61175
62-
To run the web application for development you can run
63-
<pre>
64-
$ make start-containers
65-
$ make start-worker
176+
### 4. Running the Development Server
66177
67-
# in a seperate shell
68-
$ make dev-flask
69-
</pre>
178+
```bash
179+
make dev-flask
180+
```
70181
71-
Alternatively, you can use the dockerfile with
72-
<pre>make docker && make docker-run</pre>
182+
Then open:
73183

74-
Some features like Gap Analysis require a neo4j DB running, you can start this with
75-
<pre>make docker-neo4j</pre>
76-
enviroment varaibles for app to connect to neo4jDB (default):
77-
- NEO4J_URL (neo4j//neo4j:password@localhost:7687)
184+
```
185+
http://127.0.0.1:5000
186+
```
187+
188+
---
78189

79-
To run the web application for production you need gunicorn and you can run from within the cre_sync dir
80-
<pre>make prod-run</pre>
190+
### Notes
81191

192+
* Webpack and Yarn peer-dependency warnings during installation are expected and can be safely ignored.
193+
* Some features (such as Gap Analysis) require Neo4j and may not function fully in local development environments.
82194

83-
### Using the OpenCRE API
84-
See [the myOpenCRE user guide](docs/my-opencre-user-guide.md) on using the OpenCRE API to for example add your own security guidelines and standards.
195+
---
85196

197+
## CLI Usage
86198

87-
### Docker building and running
88-
You can build the production or the development docker images with
89-
`make docker-prod` and `make docker-dev` respectively
90-
The environment variables used by OpenCRE are:
199+
```bash
200+
python cre.py --help
91201
```
92-
- name: NEO4J_URL
93-
- name: NO_GEN_EMBEDDINGS
94-
- name: FLASK_CONFIG
95-
- name: DEV_DATABASE_URL
96-
- name: INSECURE_REQUESTS # development or TLS terminated environments only
97-
- name: REDIS_HOST
98-
- name: REDIS_PORT
99-
- name: REDIS_NO_SSL
100-
- name: REDIS_URL # in case REDIS_HOST and REDIS_PORT are unavailable
101-
- name: GCP_NATIVE # if there are ambient GCP credentials, only useful for VERTEX chatbot
102-
- name: GOOGLE_SECRET_JSON # if not running on GCP
103-
- name: GOOGLE_CLIENT_ID # useful for login only
104-
- name: GOOGLE_CLIENT_SECRET # useful for login only
105-
- name: LOGIN_ALLOWED_DOMAINS # useful for login only
106-
- name: OpenCRE_gspread_Auth # useful only when importing data, possible values 'oauth' or 'service_account'
202+
203+
```bash
204+
python cre.py --review --from_spreadsheet <google_sheets_url>
205+
```
206+
207+
```bash
208+
python cre.py --add --from_spreadsheet <google_sheets_url>
209+
```
210+
211+
---
212+
213+
## Running the Web Application for Development
214+
215+
```bash
216+
make start-containers
217+
make start-worker
218+
219+
# in a separate shell
220+
make dev-flask
221+
```
222+
223+
Alternatively:
224+
225+
```bash
226+
make docker
227+
make docker-run
228+
```
229+
230+
---
231+
232+
## Neo4j (Optional)
233+
234+
```bash
235+
make docker-neo4j
236+
```
237+
238+
```
239+
NEO4J_URL=neo4j://neo4j:password@localhost:7687
240+
```
241+
242+
---
243+
244+
## Developing
245+
246+
```bash
247+
make test
248+
```
249+
250+
```bash
251+
make cover
107252
```
108-
You can run the containers with `make docker-prod-run` and `make-docker-dev-run`
109253

110-
### Developing
254+
Try to keep coverage above **70%**.
111255

112-
You can run backend tests with
113-
<pre>make test</pre>
114-
You can run get a coverage report with
115-
<pre>make cover</pre>
116-
Try to keep the coverage above 70%
256+
Code style is enforced using **Black** and **GitHub Super-Linter**.
117257

258+
---
118259

119-
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
120-
[![GitHub Super-Linter](https://github.com/OWASP/common-requirement-enumeration/workflows/Lint%20Code%20Base/badge.svg)](https://github.com/marketplace/actions/super-linter)
121-
[![Main Branch Build](https://github.com/OWASP/common-requirement-enumeration/workflows/Test/badge.svg?branch=main)](https://github.com/OWASP/OWASP/common-requirement-enumeration/workflows/Test)
260+
## License
122261

123-
[![Issues](https://img.shields.io/github/issues/owasp/common-requirement-enumeration)](https://github.com/OWASP/common-requirement-enumeration/issues)
124-
[![PR's Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat)](http://makeapullrequest.com)
125-
![GitHub contributors](https://img.shields.io/github/contributors/owasp/common-requirement-enumeration)
126-
![GitHub last commit](https://img.shields.io/github/last-commit/owasp/common-requirement-enumeration)
127-
![GitHub commit activity](https://img.shields.io/github/commit-activity/y/owasp/common-requirement-enumeration)
262+
This project is licensed under **CC0-1.0**.
128263

129-
[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://github.com/codespaces/new?hide_repo_select=true&ref=main&repo=400297709&machine=standardLinux32gb&devcontainer_path=.devcontainer%2Fdevcontainer.json&location=WestEurope)
264+
PRs welcome ❤️

0 commit comments

Comments
 (0)