Process Large CSV Exports in Background Batches – Xlinesoft Blog

Exporting hundreds of thousands of records during a normal page request can exceed PHP execution limits and leave the user waiting without any indication of progress.

This PHPRunner tutorial moves large CSV exports into a scheduled worker. Users submit an export request, the worker processes a fixed number of records per run, and the List page shows progress until the ZIP file is ready to download.

1. Creating the export queue

Create an exports table and add it to the PHPRunner project. This example supports two export types and optional date limits.


CREATE TABLE exports (
    id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
    created_at DATETIME NOT NULL,
    date_from DATE NULL,
    date_to DATE NULL,
    status VARCHAR(20) NOT NULL,
    total_records INT NOT NULL DEFAULT 0,
    processed_records INT NOT NULL DEFAULT 0,
    filename VARCHAR(255) NULL,
    completed_at DATETIME NULL,
    task_type VARCHAR(50) NOT NULL
);

On the Add page, show date_from, date_to, and task_type. Configure task_type as a Lookup wizard with the fixed values orders and transactions. Keep the worker-managed fields off the Add and Edit pages.

2. Queuing a new export

Open Exports Add page – Before record added and add:


$allowedTypes = array("orders", "transactions");

if (!in_array($values["task_type"], $allowedTypes, true)) 
    $message = "Select a valid export type.";
    return false;


$values["created_at"] = date("Y-m-d H:i:s");
$values["status"] = "Pending";
$values["total_records"] = 0;
$values["processed_records"] = 0;
$values["filename"] = null;
$values["completed_at"] = null;

return true;

The user only creates a queue record. No export data is generated during the Add request.

3. Adding the scheduled worker

Create cron.php in the generated application folder. This example uses MySQL syntax and processes up to 100,000 rows during each run. Change $recordsPerRun if the server needs smaller batches.


<?php if (PHP_SAPI !== "cli") 
    http_response_code(403);
    exit("This worker must be run from the command line.");


require_once __DIR__ . "/include/dbcommon.php";

set_time_limit(300);

$recordsPerRun = 100000;
$progressUpdateInterval = 1000;

$exportTypes = array(
    "orders" =???> array(
        "table" => "orders",
        "dateField" => "order_date",
        "headers" => array(
            "Order ID",
            "Order Date",
            "Customer",
            "Amount"
        ),
        "fields" => array(
            "id",
            "order_date",
            "customer",
            "amount"
        )
    ),
    "transactions" => array(
        "table" => "transactions",
        "dateField" => "transaction_date",
        "headers" => array(
            "Transaction ID",
            "Transaction Date",
            "Customer Name",
            "Customer Email",
            "Transaction Type",
            "Amount",
            "Status",
            "Reference"
        ),
        "fields" => array(
            "id",
            "transaction_date",
            "customer_name",
            "customer_email",
            "transaction_type",
            "amount",
            "status",
            "reference"
        )
    )
);

$rs = DB::Query(
    "SELECT * FROM exports " .
    "WHERE status IN ('Pending', 'Processing') " .
    "ORDER BY id LIMIT 1"
);
$export = $rs->fetchAssoc();

if (!$export) 
    echo "No exports to process.\n";
    exit;


$exportId = (int)$export["id"];
$taskType = trim((string)$export["task_type"]);

if (!isset($exportTypes[$taskType])) 
    DB::Update(
        "exports",
        array("status" => "Failed"),
        array("id" => $exportId)
    );
    exit("Invalid task type.\n");


$definition = $exportTypes[$taskType];
$sourceTable = $definition["table"];
$dateField = $definition["dateField"];
$alreadyProcessed = (int)$export["processed_records"];

DB::Update(
    "exports",
    array("status" => "Processing"),
    array("id" => $exportId)
);

$where = array();

if (!empty($export["date_from"])) 
    $where[] = DB::PrepareSQL(
        $dateField . " >= ':1'",
        $export["date_from"] . " 00:00:00"
    );


if (!empty($export["date_to"])) 
    $where[] = DB::PrepareSQL(
        $dateField . " <= ':1'",
        $export["date_to"] . " 23:59:59"
    );


$whereSql = count($where)
    ? " WHERE " . implode(" AND ", $where)
    : "";

if ($alreadyProcessed === 0) 
    $countRs = DB::Query(
        "SELECT COUNT(*) AS cnt FROM " . $sourceTable . $whereSql
    );
    $countData = $countRs->fetchAssoc();
    $totalRecords = (int)$countData["cnt"];

    DB::Update(
        "exports",
        array("total_records" => $totalRecords),
        array("id" => $exportId)
    );
 else 
    $totalRecords = (int)$export["total_records"];


$exportFolder = getabspath("exports");

if (!is_dir($exportFolder) && !mkdir($exportFolder, 0755, true)) 
    DB::Update(
        "exports",
        array("status" => "Failed"),
        array("id" => $exportId)
    );
    exit("Cannot create the exports folder.\n");


$csvFilename = "export_" . $exportId . ".csv";
$csvPath = $exportFolder . DIRECTORY_SEPARATOR . $csvFilename;
$file = fopen($csvPath, $alreadyProcessed === 0 ? "w" : "a");

if (!$file) 
    DB::Update(
        "exports",
        array("status" => "Failed"),
        array("id" => $exportId)
    );
    exit("Cannot open the export file.\n");


if ($alreadyProcessed === 0) 
    fputcsv($file, $definition["headers"]);


$sql = "SELECT * FROM " . $sourceTable . $whereSql;
$sql .= " ORDER BY id";
$sql .= " LIMIT " . (int)$recordsPerRun;
$sql .= " OFFSET " . (int)$alreadyProcessed;

$rs = DB::Query($sql);
$processedThisRun = 0;

while ($row = $rs->fetchAssoc()) 
    $csvRow = array();

    foreach ($definition["fields"] as $field) 
        $csvRow[] = $row[$field];
    

    fputcsv($file, $csvRow);
    $processedThisRun++;

    if ($processedThisRun % $progressUpdateInterval === 0) 
        DB::Update(
            "exports",
            array(
                "processed_records" =>
                    $alreadyProcessed + $processedThisRun
            ),
            array("id" => $exportId)
        );
    


fclose($file);

$totalProcessed = $alreadyProcessed + $processedThisRun;
$update = array("processed_records" => $totalProcessed);

if ($totalProcessed >= $totalRecords)  ZipArchive::OVERWRITE
        ) !== true
    ) 
        $update["status"] = "Failed";
        DB::Update("exports", $update, array("id" => $exportId));
        exit("Cannot create the ZIP file.\n");
    

    $zip->addFile($csvPath, $csvFilename);
    $zip->close();

    if (!file_exists($zipPath)) 
        $update["status"] = "Failed";
        DB::Update("exports", $update, array("id" => $exportId));
        exit("The ZIP file was not created.\n");
    

    unlink($csvPath);
    $update["status"] = "Completed";
    $update["filename"] = $zipFilename;
    $update["completed_at"] = date("Y-m-d H:i:s");
 else 
    $update["status"] = "Processing";


DB::Update("exports", $update, array("id" => $exportId));

echo "Processed " . $processedThisRun . " records.\n";

The worker uses an allowlist in $exportTypes, so a value stored in task_type cannot become an arbitrary table name. Each run resumes at processed_records. The status changes to ‘Completed’ only after the ZIP file has been created.

The PHP Zip extension must be enabled. The web-server account also needs write permission to the generated application’s exports folder.

4. Scheduling the worker

Schedule the generated application’s cron.php with the PHP command-line executable. For example, a Linux cron job that runs every minute can use:


* * * * * /usr/bin/php /path/to/application/cron.php

Use the equivalent scheduled-task command on Windows. Do not configure overlapping runs. The script rejects web requests, so it cannot be started by visiting its URL.

5. Displaying progress on the List page

Add status, total_records, processed_records, filename, and completed_at to the Exports List page.

Open Exports List page – JavaScript OnLoad and add:


setInterval(function() 
    if ($("div.modal-dialog").length) 
        return;
    

    location.reload();
, 10000);

The page refreshes every ten seconds, but it does not interrupt a user who has an Add, Edit, or View popup open.

6. Adding the download link

Open the filename field’s View as settings, select Custom, and add:


if ($data["status"] === "Completed" && $value) 
    $filename = basename($value);
    $url = "exports/" . rawurlencode($filename);
    $caption = htmlspecialchars(
        "Download",
        ENT_QUOTES,
        "UTF-8"
    );

    $value="'
        . $caption
        . '';
 else 
    $value = "";

Rebuild the project, submit an export request, and run the scheduled worker. The record should move from ‘Pending’ to ‘Processing’, the processed count should increase after each batch, and the download link should appear after the ZIP file is complete.

PakarPBN

A Private Blog Network (PBN) is a collection of websites that are controlled by a single individual or organization and used primarily to build backlinks to a “money site” in order to influence its ranking in search engines such as Google. The core idea behind a PBN is based on the importance of backlinks in Google’s ranking algorithm. Since Google views backlinks as signals of authority and trust, some website owners attempt to artificially create these signals through a controlled network of sites.

In a typical PBN setup, the owner acquires expired or aged domains that already have existing authority, backlinks, and history. These domains are rebuilt with new content and hosted separately, often using different IP addresses, hosting providers, themes, and ownership details to make them appear unrelated. Within the content published on these sites, links are strategically placed that point to the main website the owner wants to rank higher. By doing this, the owner attempts to pass link equity (also known as “link juice”) from the PBN sites to the target website.

The purpose of a PBN is to give the impression that the target website is naturally earning links from multiple independent sources. If done effectively, this can temporarily improve keyword rankings, increase organic visibility, and drive more traffic from search results.

Streaming Film

Nonton Film gratis

Jasa Backlink

Download Anime Batch

Process Large CSV Exports in Background Batches – Xlinesoft Blog

1. Creating the export queue

2. Queuing a new export

3. Adding the scheduled worker

4. Scheduling the worker

5. Displaying progress on the List page

6. Adding the download link

PakarPBN

By Thebespokeappcompany

You Missed

Process Large CSV Exports in Background Batches – Xlinesoft Blog

Gravity Forms MCP for nonprofits: Raise, recruit, and report with AI

Build an Equipment Checkout System with QR Code Scanning – Xlinesoft Blog

Send Gravity Forms entries to n8n

Process Large CSV Exports in Background Batches – Xlinesoft Blog

1. Creating the export queue

2. Queuing a new export

3. Adding the scheduled worker

4. Scheduling the worker

5. Displaying progress on the List page

6. Adding the download link

PakarPBN

By Thebespokeappcompany

Related Post

Gravity Forms MCP for nonprofits: Raise, recruit, and report with AI

Build an Equipment Checkout System with QR Code Scanning – Xlinesoft Blog

Send Gravity Forms entries to n8n

You Missed

Process Large CSV Exports in Background Batches – Xlinesoft Blog

Gravity Forms MCP for nonprofits: Raise, recruit, and report with AI

Build an Equipment Checkout System with QR Code Scanning – Xlinesoft Blog

Send Gravity Forms entries to n8n