Snipe-IT GetAllUsers() doesn't get all users - needs pagination support #22

Issue Description

The client.Users().GetAllUsers() function in the Snipe-IT library doesn't actually fetch all users from the system. It only retrieves the first 500
users due to hardcoded pagination parameters and lacks proper pagination implementation, resulting in missing users beyond that limit and causing API timeout issues.

Reproduction Steps

1. Set up a Snipe-IT instance with more than 500 users
2. Use the rego library to call client.Users().GetAllUsers()
3. Observe that only the first 500 users are returned
4. Try to look up a user (via email) that exists beyond user #500
5. The lookup will fail even though the user exists in Snipe-IT

  snipeClient := snipeit.NewClient(log.FATAL)
  users, err := snipeClient.Users().GetAllUsers()
  if err != nil {
      log.Fatal(err)
  }
  fmt.Printf("Retrieved %d users out of %d total", len(*users.Rows), users.Total)
  // Output: "Retrieved 500 users out of 1250 total" (missing 750 users)

Expected Behavior

• GetAllUsers() should return ALL users from the Snipe-IT system
• Should implement proper pagination to fetch users in multiple API calls
• Should handle large datasets without timeouts
• Should use reasonable page sizes (100-200 users per request) for better performance

Actual Behavior

• Only returns the first 500 users (hardcoded limit)
• Makes only a single API call with ?limit=500&offset=0
• Users beyond #500 are completely ignored
• Large page size causes API timeouts: context deadline exceeded
• Function name is misleading - it's actually GetFirst500Users()

Technical Details

Root Cause

In users.go, the GetAllUsers() function hardcodes pagination parameters:

  func (c *UserClient) GetAllUsers() (*UserList, error) {
      q := UserQuery{
          Limit:  500,  // Hardcoded limit
          Offset: 0,    // Always starts at 0, never paginated
      }
      // Makes only ONE API call, no pagination loop
  }

API Endpoints Affected

• GET /api/v1/users - Snipe-IT users endpoint
• Supports limit and offset parameters for pagination
• Example: https://ledger.snipe-it.io/api/v1/users?limit=100&offset=200

Error Messages Observed

Error fetching license list: Request Error: StatusCode=500, Method=GET, URL=https://ledger.snipe-it.io/api/v1/users, Message=Get
"https://ledger.snipe-it.io/api/v1/users?limit=500": context deadline exceeded

Impact

• Data Loss: Missing users beyond the first 500
• Timeout Issues: Large page sizes (500) cause API timeouts in production
• Failed User Lookups: FetchUserIDByEmail() fails for users not in the first 500
• Production Failures: Applications crash with "user not found" errors for valid users

Proposed Solution

Implement proper pagination in the library:

  func (c *UserClient) GetAllUsers() (*UserList, error) {
      const pageSize = 100 // Smaller page size to avoid timeouts
      var allUsers []User
      offset := 0
      var totalUsers uint32

      for {
          q := UserQuery{
              Limit:  pageSize,
              Offset: offset,
          }

          users, err := c.getUsersPage(q)
          if err != nil {
              return nil, err
          }

          if offset == 0 {
              totalUsers = users.Total
          }

          allUsers = append(allUsers, *users.Rows...)

          // Check if we got all users
          if len(*users.Rows) < pageSize || len(allUsers) >= int(totalUsers) {
              break
          }

          offset += pageSize

          // Rate limiting
          time.Sleep(500 * time.Millisecond)
      }

      return &UserList{Total: uint32(len(allUsers)), Rows: &allUsers}, nil
  }

Workaround

For immediate use, applications need to implement custom pagination:

  // Custom function that properly handles pagination
  func getAllUsersWithPagination(client *snipeit.Client, l *log.Logger) (*UserAPIResponse, error) {
      const pageSize = 100
      result := &UserAPIResponse{Rows: make([]UserStruct, 0)}
      offset := 0

      for {
          url := fmt.Sprintf("https://ledger.snipe-it.io/api/v1/users?limit=%d&offset=%d", pageSize, offset)
          // ... implement direct HTTP calls with proper pagination
          // ... handle errors and timeouts
          offset += pageSize
      }

      return result, nil
  }

Priority

High - This affects licences data integrity and causes timeout issues in concourse pipeline.

Environment

• Snipe-IT API version: Latest
• rego library: Current version
• Affected endpoints: /api/v1/users