Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ These connectors are ready to use out of the box, requiring minimal modification
- [ibm_informix_using_ibm_db](https://github.com/fivetran/fivetran_connector_sdk/tree/main/connectors/ibm_informix_using_ibm_db) - This example shows how to connect and sync data from IBM Informix using Connector SDK. This example uses the `ibm_db` library to connect to the Informix database and fetch data.
- [influx_db](https://github.com/fivetran/fivetran_connector_sdk/tree/main/connectors/influx_db) - This example shows how to sync data from InfluxDB using Connector SDK. It uses the `influxdb3_python` library to connect to InfluxDB and fetch time-series data from a specified measurement.
- [iterate](https://github.com/fivetran/fivetran_connector_sdk/tree/main/connectors/iterate) - This example shows how to sync NPS survey data from the Iterate REST API and load it into your destination using Connector SDK. The connector fetches NPS surveys and their individual responses, providing complete survey analytics data for downstream analysis.
- [jboard](https://github.com/fivetran/fivetran_connector_sdk/tree/main/connectors/jboard) - This example shows how to sync employers, job categories, and alert subscriptions data from Jboard API to your destination warehouse. You need to provide your Jboard API key for this example to work.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- [jboard](https://github.com/fivetran/fivetran_connector_sdk/tree/main/connectors/jboard) - This example shows how to sync employers, job categories, and alert subscriptions data from Jboard API to your destination warehouse. You need to provide your Jboard API key for this example to work.
- [jboard](https://github.com/fivetran/fivetran_connector_sdk/tree/main/connectors/jboard) - This example shows how to sync employers, job categories, and alert subscriptions data from the Jboard API to your destination warehouse. You need to provide your Jboard API key for this example to work.

- [leavedates](https://github.com/fivetran/fivetran_connector_sdk/tree/main/connectors/leavedates) - This example shows how to sync leave report data from LeaveDates API by using the Connector SDK. You need to provide your LeaveDates API token and company ID for this example to work.
- [microsoft_excel](https://github.com/fivetran/fivetran_connector_sdk/tree/main/connectors/microsoft_excel) - This example shows how to sync data from Microsoft Excel using Connector SDK. It shows three different ways to sync data from Excel files using `pandas`, `python-calamine` and `openpyxl`.
- [microsoft_intune](https://github.com/fivetran/fivetran_connector_sdk/tree/main/connectors/microsoft_intune/) - This example shows how to connect to Microsoft Intune and retrieve all managed devices using the Connector SDK.
Expand Down
104 changes: 104 additions & 0 deletions connectors/jboard/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# Jboard API Connector Example

## Connector overview
This connector syncs employers, job categories, and alert subscriptions data from Jboard API to your destination warehouse. The connector fetches employer profiles with their metadata, job categories with hierarchical structure, and user alert subscriptions with search criteria. It supports incremental synchronization using timestamp-based cursors and handles API rate limiting with automatic retry logic. The connector uses memory-efficient streaming to process large datasets without accumulation issues.

## Requirements
- [Supported Python versions](https://github.com/fivetran/fivetran_connector_sdk/blob/main/README.md#requirements): **3.9-3.13**
Copy link

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect Python version specification. According to the repository requirements and coding guidelines, the supported Python versions are 3.10-3.12 (with 3.13 experimental support), not "3.9-3.13" as stated here.

Change line 7 to:

- [Supported Python versions](https://github.com/fivetran/fivetran_connector_sdk/blob/main/README.md#requirements)

Remove the explicit version specification and rely on the link, or use the correct range: "3.10-3.12".

Copilot generated this review using guidance from repository custom instructions.
- Operating system:
- Windows: 10 or later (64-bit only)
- macOS: 13 (Ventura) or later (Apple Silicon [arm64] or Intel [x86_64])
- Linux: Distributions such as Ubuntu 20.04 or later, Debian 10 or later, or Amazon Linux 2 or later (arm64 or x86_64)

## Getting started
Refer to the [Connector SDK Setup Guide](https://fivetran.com/docs/connectors/connector-sdk/setup-guide) to get started.

## Features
- Syncs employer data, job categories, and alert subscriptions from Jboard API
- Bearer token authentication with secure API key handling (refer to `execute_api_request` function)
- Page-based pagination with automatic page traversal (refer to `get_employers`, `get_categories`, and `get_alert_subscriptions` functions)
- Memory-efficient streaming prevents data accumulation for large datasets using generator patterns
- Incremental synchronization using timestamp-based cursors (refer to `get_time_range` function)
- Comprehensive error handling with exponential backoff retry logic (refer to `__handle_rate_limit` and `__handle_request_error` functions)
- Configurable sync parameters including page size, timeout, and retry attempts
- Cognitive complexity optimization with helper function extraction for maintainability

## Configuration file
```json
{
"api_key": "<YOUR_JBOARD_API_KEY>",
"sync_frequency_hours": "<YOUR_JBOARD_API_SYNC_FREQUENCY_HOURS>",
"initial_sync_days": "<YOUR_JBOARD_API_INITIAL_SYNC_DAYS>",
"max_records_per_page": "<YOUR_JBOARD_API_MAX_RECORDS_PER_PAGE>",
"request_timeout_seconds": "<YOUR_JBOARD_API_REQUEST_TIMEOUT_SECONDS>",
"retry_attempts": "<YOUR_JBOARD_API_RETRY_ATTEMPTS>",
"enable_incremental_sync": "<YOUR_JBOARD_API_ENABLE_INCREMENTAL_SYNC>",
"enable_debug_logging": "<YOUR_JBOARD_API_ENABLE_DEBUG_LOGGING>"
}
```
Comment on lines +26 to +38
Copy link

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing note about not checking configuration.json into version control. According to the coding guidelines, the Configuration file section must explicitly mention that configuration.json should not be versioned.

Add after the JSON block:

Note: Ensure that the `configuration.json` file is not checked into version control to protect sensitive information.

Copilot generated this review using guidance from repository custom instructions.

### Configuration parameters
- `api_key`: Your Jboard API authentication key (required)
- `sync_frequency_hours`: How often to sync data in hours (default: 4)
- `initial_sync_days`: Days of historical data to fetch on first sync (default: 90)
- `max_records_per_page`: Maximum records per API request (default: 100, range: 1-1000)
- `request_timeout_seconds`: API request timeout in seconds (default: 30)
- `retry_attempts`: Number of retry attempts for failed requests (default: 3)
- `enable_incremental_sync`: Enable timestamp-based incremental sync (default: true)
- `enable_debug_logging`: Enable detailed debug logs (default: false)
Comment on lines +30 to +48
Copy link

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Configuration parameters documentation doesn't match configuration.json. The README documents sync_frequency_hours and enable_debug_logging as configuration parameters (lines 42, 48), but these are not actually used in the connector code (connector.py).

Either:

  1. Remove these parameters from both configuration.json and README.md documentation if they're not needed
  2. Implement the functionality for these parameters in connector.py

This inconsistency can confuse users about which parameters are actually functional.

Suggested change
"sync_frequency_hours": "<YOUR_JBOARD_API_SYNC_FREQUENCY_HOURS>",
"initial_sync_days": "<YOUR_JBOARD_API_INITIAL_SYNC_DAYS>",
"max_records_per_page": "<YOUR_JBOARD_API_MAX_RECORDS_PER_PAGE>",
"request_timeout_seconds": "<YOUR_JBOARD_API_REQUEST_TIMEOUT_SECONDS>",
"retry_attempts": "<YOUR_JBOARD_API_RETRY_ATTEMPTS>",
"enable_incremental_sync": "<YOUR_JBOARD_API_ENABLE_INCREMENTAL_SYNC>",
"enable_debug_logging": "<YOUR_JBOARD_API_ENABLE_DEBUG_LOGGING>"
}
```
### Configuration parameters
- `api_key`: Your Jboard API authentication key (required)
- `sync_frequency_hours`: How often to sync data in hours (default: 4)
- `initial_sync_days`: Days of historical data to fetch on first sync (default: 90)
- `max_records_per_page`: Maximum records per API request (default: 100, range: 1-1000)
- `request_timeout_seconds`: API request timeout in seconds (default: 30)
- `retry_attempts`: Number of retry attempts for failed requests (default: 3)
- `enable_incremental_sync`: Enable timestamp-based incremental sync (default: true)
- `enable_debug_logging`: Enable detailed debug logs (default: false)
"initial_sync_days": "<YOUR_JBOARD_API_INITIAL_SYNC_DAYS>",
"max_records_per_page": "<YOUR_JBOARD_API_MAX_RECORDS_PER_PAGE>",
"request_timeout_seconds": "<YOUR_JBOARD_API_REQUEST_TIMEOUT_SECONDS>",
"retry_attempts": "<YOUR_JBOARD_API_RETRY_ATTEMPTS>",
"enable_incremental_sync": "<YOUR_JBOARD_API_ENABLE_INCREMENTAL_SYNC>"
}

Configuration parameters

  • api_key: Your Jboard API authentication key (required)
  • initial_sync_days: Days of historical data to fetch on first sync (default: 90)
  • max_records_per_page: Maximum records per API request (default: 100, range: 1-1000)
  • request_timeout_seconds: API request timeout in seconds (default: 30)
  • retry_attempts: Number of retry attempts for failed requests (default: 3)
  • enable_incremental_sync: Enable timestamp-based incremental sync (default: true)

Copilot uses AI. Check for mistakes.

## Requirements file
This connector does not require any additional packages beyond those provided by the Fivetran environment.

Note: The `fivetran_connector_sdk:latest` and `requests:latest` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.

## Authentication
1. Log in to the [Jboard API Developer Portal](https://app.jboard.io/api/documentation).
2. Navigate to your account settings page to access API credentials.
3. Generate a new API key or copy your existing API key.
4. Make a note of the API key - it will be used as the Bearer token for authentication.
5. Use sandbox credentials for testing, production credentials for live syncing.

Note: The connector automatically handles Bearer token authentication (API key never expires). Credentials are never logged or exposed in plain text for security.

## Pagination
Page-based pagination with automatic page traversal (refer to `get_employers`, `get_categories`, and `get_alert_subscriptions` functions). The connector uses `page` and `per_page` parameters to fetch data in configurable chunks. Generator-based processing prevents memory accumulation for large datasets by yielding individual records. Processes pages sequentially while yielding individual records for immediate processing, with pagination metadata used to determine when all data has been fetched.

## Data handling
Employer, category, and alert subscription data is mapped from Jboard API's format to normalized database columns (refer to the `__map_employer_data`, `__map_category_data`, and `__map_alert_subscription_data` functions). Nested objects like tags arrays are serialized to JSON strings, and all timestamps are converted to UTC format for consistency.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Employer, category, and alert subscription data is mapped from Jboard API's format to normalized database columns (refer to the `__map_employer_data`, `__map_category_data`, and `__map_alert_subscription_data` functions). Nested objects like tags arrays are serialized to JSON strings, and all timestamps are converted to UTC format for consistency.
Employer, category, and alert subscription data is mapped from Jboard API's format to normalized database columns (refer to the `__map_employer_data`, `__map_category_data`, and `__map_alert_subscription_data` functions). Nested objects like `tags` arrays are serialized to JSON strings, and all timestamps are converted to UTC format for consistency.


Supports timestamp-based incremental synchronization using the `last_sync_time` state parameter (refer to the `get_time_range` function). Initial sync can be configured to fetch historical data up to 365 days using the `initial_sync_days` configuration parameter.

## Error handling
- 429 Rate Limited: Automatic retry with Retry-After header support and exponential backoff (refer to the `__handle_rate_limit` function)
- Timeout handling with configurable retry attempts and exponential backoff with jitter (refer to the `__handle_request_error` function)
- Exponential backoff with jitter prevents multiple clients from making requests at the same time
- HTTP error handling with appropriate retry logic for temporary failures
- Parameter validation with descriptive error messages provides clear guidance for fixing configuration issues

## Tables created
| Table | Primary Key | Description |
|-------|-------------|-------------|
| EMPLOYERS | `id` | Employer profiles with company information and metadata |
| CATEGORIES | `id` | Job categories with hierarchical structure and settings |
| ALERT_SUBSCRIPTIONS | `id` | User alert subscriptions with search criteria and preferences |

Column types are automatically inferred by Fivetran. Sample columns include:

**EMPLOYERS**: `id`, `name`, `description`, `website`, `logo_url`, `featured`, `source`, `created_at`, `updated_at`, `have_posted_jobs`, `have_a_logo`, `sync_timestamp`

**CATEGORIES**: `id`, `name`, `description`, `parent_id`, `sort_order`, `is_active`, `created_at`, `updated_at`, `sync_timestamp`

**ALERT_SUBSCRIPTIONS**: `id`, `email`, `query`, `location`, `search_radius`, `remote_work_only`, `category_id`, `job_type`, `tags`, `is_active`, `created_at`, `updated_at`, `sync_timestamp`
Comment on lines +87 to +92

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**EMPLOYERS**: `id`, `name`, `description`, `website`, `logo_url`, `featured`, `source`, `created_at`, `updated_at`, `have_posted_jobs`, `have_a_logo`, `sync_timestamp`
**CATEGORIES**: `id`, `name`, `description`, `parent_id`, `sort_order`, `is_active`, `created_at`, `updated_at`, `sync_timestamp`
**ALERT_SUBSCRIPTIONS**: `id`, `email`, `query`, `location`, `search_radius`, `remote_work_only`, `category_id`, `job_type`, `tags`, `is_active`, `created_at`, `updated_at`, `sync_timestamp`
EMPLOYERS: `id`, `name`, `description`, `website`, `logo_url`, `featured`, `source`, `created_at`, `updated_at`, `have_posted_jobs`, `have_a_logo`, `sync_timestamp`
CATEGORIES: `id`, `name`, `description`, `parent_id`, `sort_order`, `is_active`, `created_at`, `updated_at`, `sync_timestamp`
ALERT_SUBSCRIPTIONS: `id`, `email`, `query`, `location`, `search_radius`, `remote_work_only`, `category_id`, `job_type`, `tags`, `is_active`, `created_at`, `updated_at`, `sync_timestamp`


## Additional files

The connector includes several additional files to support functionality, testing, and deployment:

- `requirements.txt` – Python dependency specification for Jboard API integration and connector requirements including faker for mock testing.
Copy link

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reference to non-existent requirements.txt file. Line 98 mentions "requirements.txt – Python dependency specification for Jboard API integration and connector requirements including faker for mock testing", but no requirements.txt file exists in the connector directory.

Either:

  1. Remove this reference from the "Additional files" section if no requirements.txt is needed
  2. Add the requirements.txt file if faker or other dependencies are actually needed for testing

Based on the connector code, no additional dependencies are needed, so this reference should be removed.

Suggested change
- `requirements.txt` – Python dependency specification for Jboard API integration and connector requirements including faker for mock testing.

Copilot uses AI. Check for mistakes.

- `configuration.json` – Configuration template for API credentials and connector parameters (should be excluded from version control).


## Additional considerations
Comment on lines +94 to +103
Copy link

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect statement about requirements.txt. The README states "This connector does not require any additional packages" (line 51), but line 98 mentions "requirements.txt – Python dependency specification for Jboard API integration and connector requirements including faker for mock testing."

These statements contradict each other. If there's a requirements.txt file with faker, then the connector does have additional dependencies. Clarify which is correct and update accordingly.

Suggested change
## Additional files
The connector includes several additional files to support functionality, testing, and deployment:
- `requirements.txt` – Python dependency specification for Jboard API integration and connector requirements including faker for mock testing.
- `configuration.json` – Configuration template for API credentials and connector parameters (should be excluded from version control).
## Additional considerations
## Requirements file
The connector requires the `faker` package for mock testing and development purposes.

faker


Note: The `fivetran_connector_sdk:latest` and `requests:latest` packages are pre-installed in the Fivetran environment. To avoid dependency conflicts, do not declare them in your `requirements.txt`.

## Additional files

The connector includes several additional files to support functionality, testing, and deployment:

- `requirements.txt` – Python dependency specification for mock testing and development.
- `configuration.json` – Configuration template for API credentials and connector parameters (should be excluded from version control).

Copilot uses AI. Check for mistakes.
The examples provided are intended to help you effectively use Fivetran's Connector SDK. While we've tested the code, Fivetran cannot be held responsible for any unexpected or negative consequences that may arise from using these examples. For inquiries, please reach out to our Support team.
10 changes: 10 additions & 0 deletions connectors/jboard/configuration.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"api_key": "<YOUR_JBOARD_API_KEY>",
"sync_frequency_hours": "<YOUR_JBOARD_API_SYNC_FREQUENCY_HOURS>",
Copy link

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Configuration parameter sync_frequency_hours is defined in configuration.json but never used in the connector code. Either remove this parameter from the configuration file if it's not needed, or implement the logic to use it.

Note: Fivetran manages sync frequency at the platform level, so this parameter may be unnecessary for connector logic.

Suggested change
"sync_frequency_hours": "<YOUR_JBOARD_API_SYNC_FREQUENCY_HOURS>",

Copilot uses AI. Check for mistakes.
"initial_sync_days": "<YOUR_JBOARD_API_INITIAL_SYNC_DAYS>",
"max_records_per_page": "<YOUR_JBOARD_API_MAX_RECORDS_PER_PAGE>",
"request_timeout_seconds": "<YOUR_JBOARD_API_REQUEST_TIMEOUT_SECONDS>",
"retry_attempts": "<YOUR_JBOARD_API_RETRY_ATTEMPTS>",
"enable_incremental_sync": "<YOUR_JBOARD_API_ENABLE_INCREMENTAL_SYNC>",
"enable_debug_logging": "<YOUR_JBOARD_API_ENABLE_DEBUG_LOGGING>"
Comment on lines +3 to +9
Copy link

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Placeholder descriptions are too verbose and redundant. According to the coding guidelines, placeholders should be clear and concise, describing what the user needs to provide without excessive prefixes.

Change:

  • <YOUR_JBOARD_API_SYNC_FREQUENCY_HOURS><SYNC_FREQUENCY_IN_HOURS>
  • <YOUR_JBOARD_API_INITIAL_SYNC_DAYS><INITIAL_SYNC_DAYS>
  • <YOUR_JBOARD_API_MAX_RECORDS_PER_PAGE><MAX_RECORDS_PER_PAGE>
  • <YOUR_JBOARD_API_REQUEST_TIMEOUT_SECONDS><REQUEST_TIMEOUT_SECONDS>
  • <YOUR_JBOARD_API_RETRY_ATTEMPTS><RETRY_ATTEMPTS>
  • <YOUR_JBOARD_API_ENABLE_INCREMENTAL_SYNC><TRUE_OR_FALSE>
  • <YOUR_JBOARD_API_ENABLE_DEBUG_LOGGING><TRUE_OR_FALSE>

The excessive "YOUR_JBOARD_API_" prefix doesn't add value and makes the placeholders harder to read.

Copilot generated this review using guidance from repository custom instructions.
Comment on lines +8 to +9
Copy link

Copilot AI Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Configuration parameter enable_debug_logging is defined in configuration.json but never used in the connector code. Either remove this parameter from the configuration file if it's not needed, or implement debug logging logic in the connector.

Suggested change
"enable_incremental_sync": "<YOUR_JBOARD_API_ENABLE_INCREMENTAL_SYNC>",
"enable_debug_logging": "<YOUR_JBOARD_API_ENABLE_DEBUG_LOGGING>"
"enable_incremental_sync": "<YOUR_JBOARD_API_ENABLE_INCREMENTAL_SYNC>"

Copilot uses AI. Check for mistakes.
}
Loading
Loading