Weird (really weird) situation

Need Help
1

Hi everyone,
Today I faced a really weird situations. I have an API built in CakePHP 3.7. That API receives requests from:

  1. 10 tablets using an Android Java app built by me
  2. A lot of info from an ERP software. I built the integration with the ERP too, in Java

This is a really concise API, consisting of 12 entities, but some of them have plenty of records. The most massive has 3.000.000 records, and the second one has 300.000. The database is a MySQL

So, a couple months ago we detected some of the records from the app didn’t upload correctly. This is a simple record, with six fields, most of them integers and one text field. So, I implemented a request-response special log and sent an update to the tablets, so next time it was detected we could know what happened.

To my surprise, when we detect the case, we can see that the API received the message, and gave a 200 OK response with a record having an ID, so it should be stored. When I went to the database, the record was missing.

Now I’ve implemented a post synch control, in order to retry the false oks. But it makes me wonder, has anyone experienced this issue, where CakePHP 3 returns a non-error entity after save but it hasn’t really persisted?

I am waiting to gain permissions to MySQL logs in order to get to know the core issue. Meanwhile, I could gather information and there are about 9.000 records missing on the 300.000 records table.

Saludos,

2

Can you show the code where you create and save the entity?

3

Hi Corey,
Yes, this is the code:

public function add()
{
    $actaArticulo = $this->ActaArticulos->newEntity();
    if ($this->request->is('post')) {
        $actaArticulo = $this->ActaArticulos->patchEntity($actaArticulo, $this->request->data);
        if ($this->ActaArticulos->save($actaArticulo)) {
        	$return = ['code' => '1', 'msg' => 'Articulo ingresado', 'item' => $actaArticulo];
            $this->Flash->success(__('The {0} has been saved.', 'Acta Articulo'));
            if($this->request->getParam('_ext', '') != 'json') {
            	return $this->redirect(['action' => 'index']);
            }
        } else {
            $return = ['code' => '2', 'msg' => 'No se pudo ingresar el detalle', 'errors' => $actaArticulo->getErrors(), 'item' => $actaArticulo];
            $this->Flash->error(__('The {0} could not be saved. Please, try again.', 'Acta Articulo'));
        }
    }
    if($this->request->getParam('_ext', '') != 'json') {
        $actas = $this->ActaArticulos->Actas->find('list', ['limit' => 200]);
        $articulos = $this->ActaArticulos->Articulos->find('list', ['limit' => 200]);
        $chequeos = $this->ActaArticulos->Chequeos->find('list', ['limit' => 200]);
    } else {
        $actas = [];
        $articulos = [];
        $chequeos = [];
    }        
    $this->set('return', $return);
    $this->set(compact('actaArticulo', 'actas', 'articulos', 'chequeos'));
    $this->set('_serialize', ['return']);
}

It’s basically the standard add method.

Saludos,

4

Any chance at all that the records are being added, but then deleted later? Otherwise, I’d maybe start wondering about things like hard drive instability, because this seems very straight-forward.

5

I have experienced unexpected request behavior from mobile clients esp. related to app resume which could result in a weird outcome, but never a data loss. CakePHP was always solid, I e.g. just wasn’t expecting the same request three milliseconds apart, which was faster than MySQL could persist the first request.

Log as much as you can at the appropriate points during the request. At the end of the request the data was either written or not, logging should provide hints what’s going on.

6

Hi!
Thank you all for your points of view.
Since I don’t manage the server where the app is installed, I’m waiting for the sysadmin to give me permissions to mysql logs.
I’ll take the suggestion to log the transaction a little more.

The API doesn’t provide any delete feature for any table (it’s an append-only API) and it is not deleting elsewhere.

7

With only 10 clients I assume performance isn’t an issue at all in the sense that you aren’t hammered by requests.

If you have at least file-access on the server to access Cake’s log files I would:

  1. Log the last created DB entry in the Controller’s shutdown for that action
  2. Enable SQL query logging for that action [1] (which you normally shouldn’t do in production, but with only 10 clients …)

Work backwards from that.

[1] https://book.cakephp.org/3/en/orm/database-basics.html#query-logging

1 Like