Fixing Performance issue of team page by Adding cache in 3.x

[shailesh-google]

Fixing Performance issue of team page by Adding cache

Partial Fix for #1150

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 44.24%. Comparing base (8227260) to head (226d67d).

Additional details and impacted files

@@            Coverage Diff            @@
##                3.x    #1210   +/-   ##
=========================================
  Coverage     44.24%   44.24%           
  Complexity     3070     3070           
=========================================
  Files           344      344           
  Lines         11235    11235           
=========================================
  Hits           4971     4971           
  Misses         6264     6264           

Files with missing lines	Coverage Δ
...igee_edge_teams/src/Entity/Storage/TeamStorage.php	61.53% <100.00%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[divya-intelli]

Hi @boobaa ,

This PR partially optimizes the team listing. Team data is cached on the first load. For all subsequent requests, the data is retrieved from the cache, leading to a significant performance gain—we observed a nearly 50% reduction in load time starting from the second load.

Next Step:Optimizing Data Rendering in Drupal, which we've identified as the next major area for speed improvement.

[boobaa]

Hi @divya-intelli ,

Sounds like a good first step. As a test, I have a local site with drupal/apigee_edge:3.0.13 which is configured to fetch ALL the Apigee-related data for the /teams page from a local endpoint (to eliminate both the network latency AND the time Apigee Edge/OPDK would need to collect and return the data), with some 10k (!) teams.

• Without any patches, 10k /teams page load time: 11.1/8.4 secs (cold/warm caches)
• With this PR applied, 10k /teams page load time: 12.1/7.2 secs (which is interesting, but as it decreases the time needed for warm caches, it should be OK)
• With this PR and a custom patch (that uses the NullBackend for TeamStorage's cache) also applied, 10k /teams page load time: 9.7/5.5 secs

So while this seems to be a good first step, further optimization is still needed. Whether it's regarding data rendering or something else in Drupal, I'm not sure. Some profiling could help identifying the most painful parts in the code, but I didn't have the bandwidth for that.

[shailesh-google]

Thanks @boobaa for testing this PR.

My investigation into the latency issue identified that the primary cause was the time consumed by data rendering. Consequently, pagination was adopted as the optimal solution to effectively address and optimize this rendering constraint.

We have successfully implemented Drupal pagination ( PR #1212 ) to address this, resulting in a 50% reduction in initial page load time. By combining this with caching logic (PRs #1212 and #1210 ), subsequent page loads (2nd, 3rd, and beyond) are now seeing an even greater performance boost a massive 80% to 85% reduction in load time, as all subsequent pagination data is served from the cache.