Optimizing Laravel database queries — 1st tip : Using Chunk()

If your application is running slow or making a lot of database queries, follow the below performance optimization tips to improve your application loading time.

I shared tips for optimizing your MySQL, eloquent and raw database queries, And in this topic I’m gonna explain the first common tip,

1. Retrieving large datasets

This tip mainly focuses on improving the memory usage of your application when dealing with large datasets.

If your application needs to process large set of records, instead of retrieving all at once, you can retrieve a subset of results and process them in groups.

To retrieve a large set of results from a table called posts, we would usually do like below.

$posts = Post::all(); // when using eloquent $posts = DB::table('posts')->get(); // when using query builder 
foreach ($posts as $post)
{
   // Process posts 
}

The above examples will retrieve all the records from the posts table and process them. What if this table has 1 million rows? We will easily run out of memory.

To avoid issues when dealing with large datasets, we can retrieve a subset of results and process them as below.

- Using chunk

// when using eloquent 
$posts = Post::chunk(100, function($posts){
             foreach ($posts as $post){
                // Process posts
             } 
           });
// when using query builder
$posts = DB::table('posts')
          ->chunk(100, function ($posts){
             foreach ($posts as $post){
                 // Process posts
              } 
           });

The above example retrieves 100 records from the posts table, processes them, retrieves another 100 records and processes them. This iteration will continue until all the records are processed.

This approach will make more database queries but highly memory efficient. Usually processing of large datasets will be done in the background. So it is ok to make more queries when running in the background to avoid running out of memory when processing large datasets.