Building a custom WooCommerce product importer with advanced CSV handling and automation

Issue summary:
A large eCommerce site required a robust, flexible product importer capable of updating thousands of WooCommerce products at once from complex CSV files. The Freshy development team engineered a custom Product Information Management (PIM) importer that supports advanced data mapping, custom fields, error handling, and automatic resumption of large imports without server timeouts.

Issue background

The client’s catalog included tens of thousands of SKUs with complex metadata such as product attributes, manufacturer hierarchies, volume discount tiers, and technical PDFs. Their old “PIM Importer” tool could no longer handle the growing volume or diversity of product data.

The new importer needed to:

• Support both new product creation and updates to existing SKUs.
• Handle special characters in descriptions and titles.
• Process large CSV uploads (sometimes thousands of SKUs).
• Import images, PDFs, and videos via URL.
• Map complex taxonomies for Manufacturers, Brands, and Categories.
• Automatically resume long-running imports that exceed server time limits.
• Provide detailed logs showing what changed after each run.

Diagnosis

The old importer had several limitations:

• It would timeout when processing large CSVs.
• It duplicated SKUs if the same file ran more than once.
• It broke on special characters such as m-dashes or smart quotes.
• It lacked detailed logs and rollback visibility.

The Freshy team determined the best path forward was to build a new PIM Importer plugin leveraging native WooCommerce APIs but optimized for large-batch imports and incremental updates.

Resolution steps

1. CSV logic and flexible field mapping
   The importer uses SKU as the unique key for all operations. Any column can be included or omitted — missing columns leave data unchanged. A special token __DELETE__ allows bulk removal of field data. CSVs can include hundreds of columns, but partial imports (e.g., SKU + price) still work. Supports all product fields, attributes, and custom metadata.
2. Support for custom fields and taxonomies
   Added mappings for PDP Family ID and Sequence, Use Group, Manufacturer and Brand (hierarchical taxonomy), Prop65, Model Number, Case Pack, Case Only, Weight, and Volume Discounts. The importer ensures terms exist before assigning them, creating parent/child relationships automatically.
3. File handling and scheduling
   Upload CSV files to /wp-content/uploads/pim-inbox/. Processed files move to /wp-content/uploads/pim-processed/. Logs and reports are stored under /wp-content/uploads/pim-logs/. Imports run manually via a secure URL using a unique token to authenticate requests. The importer supports both run=1 (safe mode) and force=1 (immediate start) flags.
4. Automatic resumption and cron scheduling
   Large imports are split into chunks. Each chunk processes within a 60-second time window. When a chunk completes, a temporary cron schedules the next run within 60 seconds. If site traffic slows, an external heartbeat (UptimeRobot) triggers WP Cron to ensure imports continue running reliably overnight.
5. Enhanced error handling and encoding normalization
   Added automatic UTF-8 normalization to clean Windows-style characters and em-dashes. Logs now record “before” and “after” states for each SKU to ensure changes can be validated. CSV logs include full text previews for any fields affected by special character replacement.
6. Cross-sells and relational data
   Added new support for cross-sell relationships via a crosssell column. Accepts either SKUs or product IDs, separated by semicolons. Supports __DELETE__ to clear existing cross-sells.
7. Performance optimization and testing
   The importer automatically clears transients and regenerates product caches. Validated large imports with more than 5,000 SKUs without duplication. Confirmed WooCommerce PDPs reflect new data in real-time. Tested resume reliability under low-traffic conditions.

Final outcome

The new PIM Importer successfully automates all product imports for the client’s WooCommerce store. It can now process thousands of SKUs per batch, resume automatically if paused, and maintain full visibility into what changed.

Other benefits include:

• Stable imports without manual database intervention.
• Flexible column support for incremental updates.
• Seamless handling of special characters.
• Hierarchical taxonomy assignment for manufacturers and brands.
• Reliable cross-sell and relationship updates.

The system has dramatically reduced manual catalog work, allowing the client’s content team to focus on quality and speed instead of data entry troubleshooting.

Key takeaways

• When importing large WooCommerce catalogs, incremental processing with resumable chunks is critical.
• Always normalize CSV input to UTF-8 to prevent text corruption.
• Implement strong logging and rollback tools for auditing large imports.
• Custom field support and flexible mappings save time and reduce dependency on manual updates.
• Using an external heartbeat service ensures WP Cron tasks continue reliably during off-hours.

If your WooCommerce store needs a custom importer that can handle complex data structures and large product catalogs, contact the Freshy team to build a scalable, automated import solution tailored to your site.