JobScout

JobScout 🚀

Description

JobScout is a backend application built with NestJS that scrapes job postings from itviec.com and stores them in a PostgreSQL database. It provides an API to fetch these job listings.

Table of Contents

• 🚀 Project Title & Badges
• 📝 Description
• 📚 Table of Contents
• ✨ Features
• 🛠️ Tech Stack
• 🚀 Installation
• 💡 Usage
• ✨ Footer

✨ Features

• Job Scraping: Scrapes job data from itviec.com based on a provided keyword.
• Data Persistence: Stores scraped job information in a PostgreSQL database.
• API Endpoints: Provides RESTful API endpoints to retrieve job listings.
• Automated Data Collection: Supports automated collection of job data.
• TypeORM Integration: Utilizes TypeORM for efficient database interactions and migrations.
• Environment Configuration: Uses dotenv for managing environment variables.

🛠️ Tech Stack

• Language: TypeScript
• Framework: NestJS
• Database: PostgreSQL
• Scraping: Cheerio, Axios
• ORM: TypeORM
• Containerization: Docker (with Docker Compose)
• Development Tools: ESLint, Prettier, Jest

🚀 Installation

Prerequisites

• Node.js (v20.0.0 or higher)
• npm or pnpm (v10.0.0 or higher)
• PostgreSQL Database

Steps

1. Clone the repository:

   git clone https://github.com/phambaophuc/JobScout.git
   cd JobScout
   

2. Install dependencies:

   npm install
   # or
   pnpm install
   

3. Configure environment variables:
   Create a .env file in the root directory based on .env.example:

   cp .env.example .env
   

   Edit the .env file with your PostgreSQL database credentials:

   DB_HOST=localhost
   DB_PORT=5432
   DB_USERNAME=your_db_username
   DB_PASSWORD=your_db_password
   DB_DATABASE=job_finder
   

4. Run database migrations:
   This will create the necessary tables in your PostgreSQL database.

   npm run migration:run
   # or
   pnpm migration:run
   

5. Build the project:

   npm run build
   # or
   pnpm build
   

6. Run the application:

   # Development mode (with watch)
   npm run start:dev
   # or
   pnpm start:dev
   
   # Production mode
   npm run start:prod
   # or
   pnpm start:prod
   

💡 Usage

Scraping Jobs

You can trigger a job scraping process by sending a POST request to the /insert endpoint with a k query parameter for the job keyword.

Example using curl:

curl -X POST "http://localhost:3000/insert?k=nestjs"

This will scrape jobs related to "nestjs" from itviec.com and store them in the database.

Fetching Job Listings

You can retrieve all scraped job listings by sending a GET request to the /jobs endpoint.

Example using curl:

curl -X GET "http://localhost:3000/jobs"

This will return a JSON array of all job postings currently stored in the database.

✨ Footer

© 2023 JobScout. All rights reserved.

Back to Top ⬆️

---