> ## Documentation Index
> Fetch the complete documentation index at: https://docs.siftstack.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Recover failed ingestion data

> Review DLQ entries, download failed data, and reingest it after resolving the issue.

export const MintTable = ({columns = [], rows = [], columnWidths = []}) => {
  const pushTextWithLineBreaks = (parts, text, keyBase) => {
    const segments = String(text).split(/\\n|\n/);
    segments.forEach((segment, idx) => {
      if (segment) {
        parts.push(<span key={`${keyBase}-text-${idx}`}>{segment}</span>);
      }
      if (idx < segments.length - 1) {
        parts.push(<br key={`${keyBase}-br-${idx}`} />);
      }
    });
  };
  const parseMarkdown = text => {
    if (text === null || text === undefined) return "";
    const str = String(text);
    const parts = [];
    let lastIndex = 0;
    const pattern = /(`[^`]+`|\*\*[^*]+\*\*|\*[^*]+\*|\[([^\]]+)\]\(([^)]+)\))/g;
    let match;
    while (true) {
      match = pattern.exec(str);
      if (match === null) {
        break;
      }
      if (match.index > lastIndex) {
        pushTextWithLineBreaks(parts, str.substring(lastIndex, match.index), `before-${lastIndex}`);
      }
      const fullMatch = match[0];
      if (fullMatch.startsWith("`") && fullMatch.endsWith("`")) {
        parts.push(<code key={match.index}>{fullMatch.slice(1, -1)}</code>);
      } else if (fullMatch.startsWith("**") && fullMatch.endsWith("**")) {
        parts.push(<strong key={match.index}>{fullMatch.slice(2, -2)}</strong>);
      } else if (fullMatch.startsWith("*") && fullMatch.endsWith("*")) {
        parts.push(<em key={match.index}>{fullMatch.slice(1, -1)}</em>);
      } else if (fullMatch.startsWith("[")) {
        const linkText = match[2];
        const linkUrl = match[3];
        parts.push(<a key={match.index} href={linkUrl} className="text-black-600 dark:text-black-400">
            {linkText}
          </a>);
      }
      lastIndex = pattern.lastIndex;
    }
    if (lastIndex < str.length) {
      pushTextWithLineBreaks(parts, str.substring(lastIndex), `tail-${lastIndex}`);
    }
    if (parts.length > 0) {
      return parts;
    }
    const plainParts = [];
    pushTextWithLineBreaks(plainParts, str, "plain");
    return plainParts.length ? plainParts : str;
  };
  const safeColumns = Array.isArray(columns) ? columns : [];
  const safeRows = Array.isArray(rows) ? rows : [];
  const safeColumnWidths = Array.isArray(columnWidths) ? columnWidths : [];
  const hasColumnWidths = safeColumnWidths.some(w => w !== null && w !== undefined && w !== "");
  const toCssWidth = width => typeof width === "number" ? `${width}px` : String(width);
  const getColumnStyle = idx => {
    const rawWidth = safeColumnWidths[idx];
    if (rawWidth === null || rawWidth === undefined || rawWidth === "") {
      return undefined;
    }
    const width = toCssWidth(rawWidth);
    return {
      width,
      minWidth: width
    };
  };
  const containerStyle = hasColumnWidths ? undefined : {
    overflowX: "auto"
  };
  const tableStyle = hasColumnWidths ? {
    tableLayout: "fixed",
    width: "100%"
  } : {
    width: "max-content",
    minWidth: "100%"
  };
  if (!Array.isArray(columns) || !Array.isArray(rows) || !Array.isArray(columnWidths)) {
    console.warn("MintTable received invalid props:", {
      columns,
      rows,
      columnWidths
    });
  }
  if (!safeColumns.length && !safeRows.length) {
    return null;
  }
  return <div className="mint-table-container" style={containerStyle}>
      <table style={tableStyle}>
        {hasColumnWidths && <colgroup>
            {safeColumns.map((_, idx) => {
    const style = getColumnStyle(idx);
    return <col key={idx} style={style} />;
  })}
          </colgroup>}
        <thead>
          <tr>
            {safeColumns.map((col, idx) => <th key={idx} className="text-left" style={getColumnStyle(idx)}>
                <b>{parseMarkdown(col)}</b>
              </th>)}
          </tr>
        </thead>
        <tbody>
          {safeRows.map((row, rIdx) => {
    const safeRow = Array.isArray(row) ? row : [];
    return <tr key={rIdx}>
                {safeRow.map((cell, cIdx) => <td key={cIdx} style={getColumnStyle(cIdx)}>
                    {parseMarkdown(cell)}
                  </td>)}
              </tr>;
  })}
        </tbody>
      </table>
    </div>;
};

When data fails to ingest into Sift, the error is logged in the Dead Letter Queue (DLQ) and surfaced in the **Data Processing** page of the Manage workspace. From here you can review DLQ entries, download the data along with the reasons why it failed, resolve the issue, and reingest the data.

## Before you begin

* You are familiar with how [Sift organizes data into Assets, Channels, and Runs](/documentation/get-started/data-model).

## How Data Processing works

Data processing errors are logged in the Dead Letter Queue (DLQ) when ingestion fails due to invalid values or parameters. These errors can be reviewed by navigating to the Manage section and selecting **Data Processing** in the Sift interface.

The following table describes the most common error types:

<MintTable
  columns={['Error type', 'Description']}
  columnWidths={['35%', '65%']}
  rows={[
['Invalid numerical values', '`NaN` or `Infinity` values submitted in the data payload'],
['Missing or incorrect config ID', '`ingestionConfigId` values that are absent or improperly specified'],
['Invalid flow name', 'Flow names that are not recognized or formatted incorrectly'],
]}
/>

The following table describes how long error data is retained in Sift:

<MintTable
  columns={['Data', 'Retention period']}
  columnWidths={['40%', '60%']}
  rows={[
['Error records', '31 days'],
['DLQ parquet files', '46 days. Automatically deleted using a lifecycle policy.'],
]}
/>

## Review and download DLQ data

1. Click your profile icon.
2. Select **Manage**.
3. In the **Manage** navigation menu, click **Data Processing**.
4. In the **Data Processing** table, locate the error record you want to investigate.
5. Click **Export** to download the data and the reasons why it was sent to the DLQ.
6. Review the downloaded data to understand the issue.
7. Resolve the issue independently or work with Sift to resolve it, then reingest the data.

## Recover data lost during a streaming interruption

A brief network interruption during streaming ingestion is not always retried indefinitely. The streaming ingestor's retry policy uses exponential backoff: if a connection cannot reconnect quickly, retry delays reach the multi-second range, and data for that window can be permanently dropped instead of automatically backfilled.

To recover a gap caused by a streaming interruption, export the affected time range (for example, as a Parquet file) and reimport it. See [Import telemetry from a file](/documentation/ingest/data-import/import-data-from-a-file).

## Next steps

* To monitor DLQ metrics as time-series data, see [Monitor ingestion health](/documentation/ingest/monitor-ingestion-health).