Two numbers decide whether a large import finishes: how long your server lets one process run, and how much memory that process may hold. Everything else on this page exists to help you work out which of the two stopped you.
The batch size is not one of the numbers you can change. It is fixed at 25 listings per request, and there is no setting for it.
How a manual import runs
Clicking start on an import task queues one background job and returns. The job then does all of the work in a single PHP process.
- It works out how many listings the task’s filters found, and how many of those you asked for.
- It plans the run as a list of requests of 25 listings each.
- It fetches a page, writes those listings, then frees memory before the next page.
- After every page it updates the progress counters the screen reads, and writes a log line with the batch number and the memory in use.
Because it is one process, the server’s limits apply to the whole task rather than to each page. A task that needs twenty minutes cannot be run on a server that stops processes after sixty seconds, no matter how small each page is.
The 10,000 ceiling
A single import task will not import more than 10,000 listings. If the filters find more, the task quietly plans for 10,000 and reports that as its target rather than raising an error.
A feed larger than that has to be split across several import tasks with filters that do not overlap. That is also the answer for a feed well under 10,000 that your server cannot finish in one go.
The memory figure on each import log line is the fastest triage you have. Memory that stays roughly flat across batches and then stops means the process ran out of time. Memory that climbs batch after batch and then stops means it ran out of memory. The two need different fixes and they look identical from the import screen.
How the hourly sync differs
The hourly sync is not one job per task. It is a single scheduled run that walks every import task flagged for auto update and processes them one after another inside the same process, reading the task list 100 at a time to keep its own memory use small.
Two things follow from that. The sync is always the largest single job on the site, so a server that copes with individual manual imports can still fail the sync. And a run is protected by a lock lasting fifteen minutes, so an overlapping schedule cannot start a second copy and process the same listings twice.
The sync also exits immediately, without writing anything, when there is no valid API token or the MLS connection test has not passed. An empty cron log for today can therefore mean the connection, not the schedule.
If manual imports finish and the hourly sync fails, do not raise limits first. Take some tasks off auto update so the sync has less to do in one run, confirm that fixes it, and then decide whether the hosting needs changing. It is a five minute test that tells you whether size is the problem.
Keeping large feeds manageable
- Split by a stable filter. City, property type and status divide a feed into groups that do not move around. Price bands do not, because a price change moves a listing between tasks.
- Import fewer fields. Every selected field is another value written per listing. The Select Import Fields screen is where that is decided.
- Do the first import once, then rely on the sync. The first run writes everything. After that the sync only has changed listings to deal with, which is a far smaller job.
- Leave reconciliation alone. It runs once a day, separately, and removes listings that have left the MLS. It is not part of the import load.
Frequently asked questions
Can I change the number of listings per batch?
No. The page size is fixed in the plugin, and the size of an import task is controlled by its filters instead.
Why does the memory figure go up and down within one run?
The plugin releases memory between batches on purpose. A sawtooth pattern is healthy. A line that only climbs is the one to worry about.
Can two import tasks run at the same time?
They can be queued at the same time, and on a server that is already at its limit that makes both slower and more likely to be stopped. Run large tasks one at a time.
Do listing photos count against the memory limit?
Barely. Photos stay on the MLS image source and are referenced rather than downloaded, so what consumes memory is the listing data and the WordPress writes, not the images.
Related
- Hosting requirements and recommendations
- Imports that stop at 25 listings
- Turning on logging and reading the log files
- Avoiding overlapping import tasks
- How the hourly auto update works
Send us the last few lines of an import log and we will tell you which limit stopped the run.