Copy huge amount of data in Cake style?


#1

Hi,
How to do this query in Cake style like:
INSERT INTO tbl1 (fld_id)
SELECT tbl2.fld_order_id
FROM tbl2
WHERE tbl2.fld_order_id > 100;

I know two methods:
1.$data = $this->Model2->find(‘all’, … );
and $this->Model1->saveAll($data);

  • Not suitable , if I need more than 100k records (requred spetial setting for memory XXGiga bytes)
  1. $this->Model->query( INSERT …); It’s not Cake style :slight_smile:
    I have to create conditions on fly, and in that case Cake stle very helpful

Ideas?

Thank you.


#2

You can use Shells for that, load the models and leave it running in the CLI


#3

Yes I use Shells as well for this kind of “big data” :wink:


#4

Some times I use CLI too. But I have situation when a user has to run that query.
Long way :

  1. A user create task “INSERT”
  2. Cron run it as CLI.

#5

You can use a queue system.
For example cakephp-queue so that an user adds a jobs, and a constant cronjob waits for it and run it.


#6

ruul338>
Thank you.
It’s strange I see few queue systems , but didn’t see that.
I use gearman with small mysql queue system.