ℹ️ For new users, we recommend using our native Elasticsearch tools, rather than the standalone Workplace Search product. We are actively developing new features and capabilities in the Elastic Stack to help you build powerful search applications. Refer to this blog post for more information.
« Troubleshooting Common source synchronization issues »
Elastic Docs › ›Troubleshooting

Troubleshooting errors during content source sync

edit

Troubleshooting errors during content source sync

edit

It is possible to download diagnostics data about the content source and its recently completed sync jobs. There is a button to do that on content source’s Settings page:

Downloading diagnostics data.
Figure 139. Downloading diagnostics data

The data is available in JSON format. It contains some general diagnostics information, and then more detailed information for the 3 most recent sync jobs of every type. For every job, this may include:

  1. Date and time of creation
  2. Date and time of completion
  3. Duration
  4. Status
  5. Fatal exception
  6. Error reason
  7. Number of total document errors that happened during sync
  8. Detailed information on 20 last document errors

Example:

{
  "general": {
    "version": {
      "number": "7.13.0",
      "build_hash": "abc",
      "build_date": null
    },
    "elasticsearchVersion": "7.11.0"
  },
  "source": {
    "id": "xyz",
    "serviceType": "confluence_server",
    "name": "Confluence (Server)",
    "context": "organization",
    "dlpEnabled": false,
    "recentJobs": {
      "full": [
        {
          "createdAt": "2021-03-05T19:14:59Z",
          "status": "complete",
          "fatalException": {
	      "friendly_message": "Exceeded maximum consecutive errors",
	      "stack_trace": "/path/to/code/class_x.rb:5\nfrom /path/to/code/class_y.rb:10\nfrom /path/to/code/class_z.rb:15",
	      "id": "ade324cdfef",
	      "message": "Exceeded maximum consecutive errors - saw 11 errors in a row. Tripped by - RuntimeError: Oh snap, Document id:aeTpgRLAIiAAAAAAAAAAfQ is bad news",
	      "class": "Connectors::MaxSuccessiveErrorsExceededError"
	  },
          "errorReason": null,
          "completedAt": "2021-03-05T19:15:09Z",
          "durationSeconds": 10,
          "documentErrorCount": 0,
          "documentErrors": []
        }
      ],
      "incremental": [
        {
          "createdAt": "2021-03-05T19:14:52Z",
          "status": "error",
          "fatalException": null,
          "errorReason": null,
          "completedAt": "2021-03-05T19:14:56Z",
          "durationSeconds": 4,
          "documentErrorCount": 1,
          "documentErrors": [
            {
              "error_id": "6077220e4993c817",
	            "error_class": "ExampleError",
              "error_message": "Could not extract document",
              "stack_trace": "/path/to/code/class_x.rb:5\nfrom /path/to/code/class_y.rb:10\nfrom /path/to/code/class_z.rb:15"
            }
          ]
        }
      ],
      "delete": [
        {
          "createdAt": "2021-03-05T19:20:36Z",
          "status": "complete",
          "fatalException": null,
          "errorReason": null,
          "completedAt": "2021-03-05T19:20:42Z",
          "durationSeconds": 6,
          "documentErrorCount": 0,
          "documentErrors": []
        }
      ]
    },
    "oauthTokenAgeSeconds": 50000,
    "oauthScopes": null
  }
}

Troubleshoot using logs

app-server.log and connectors.log are two useful log files to review as part of troubleshooting any content source sync issues.

Use the log_level setting to produce more debug-level logging if necessary. Refer to Logs and logging in the Enterprise Search documentation for more information.

« Troubleshooting Common source synchronization issues »