B-Tree in-place node writes #227

New issue

Closed

opened 2021-03-10 10:43:56 +00:00 by jamie · 3 comments

jamie commented

2021-03-10 10:43:56 +00:00

(Migrated from git.hazaar.io)

Problem Statement

When using B-Tree as cache files, the file cache supports keepalive, which means whenever a value is read from the cache, the cache timeout value is updated. This causes a write-append to the B-Tree file with the new node. Consequently, even with only read operations, the file size continues to grow. Although the increase is just the size of the node that stores __cache_timeout, if this node contains large data, multiple unnecessary copies are stored on every write.

Who will benefit?

This feature will benefit developers and users who rely on B-Tree as cache files by preventing unnecessary file size increases and improving overall performance and efficiency.

Benefits and risks

Benefits

Prevents the B-Tree cache file from growing indefinitely with read operations.
Improves the efficiency and performance of the caching mechanism.
Reduces disk space usage by eliminating unnecessary data duplication.

Risks

In-place writes may introduce complexity and require careful handling to ensure data integrity.
Thorough testing is needed to ensure that the changes do not introduce new bugs or performance issues.

Proposed solution

Implement in-place writes for the B-Tree cache files. If the size of the data being written is the same or smaller than the previous data, the node should be written to the same location in the file. This can be achieved around lines #371 to #375 of Btree.php, which is where the serialized write occurs.

Examples

N/A

Priority/Severity

High (This will bring a huge increase in performance/productivity/usability/legislative cover)
Medium (This will bring a good increase in performance/productivity/usability)
Low (anything else e.g., trivial, minor improvements)

## Problem Statement When using B-Tree as cache files, the file cache supports keepalive, which means whenever a value is read from the cache, the cache timeout value is updated. This causes a write-append to the B-Tree file with the new node. Consequently, even with only read operations, the file size continues to grow. Although the increase is just the size of the node that stores `__cache_timeout`, if this node contains large data, multiple unnecessary copies are stored on every write. ## Who will benefit? This feature will benefit developers and users who rely on B-Tree as cache files by preventing unnecessary file size increases and improving overall performance and efficiency. ## Benefits and risks ### Benefits - Prevents the B-Tree cache file from growing indefinitely with read operations. - Improves the efficiency and performance of the caching mechanism. - Reduces disk space usage by eliminating unnecessary data duplication. ### Risks - In-place writes may introduce complexity and require careful handling to ensure data integrity. - Thorough testing is needed to ensure that the changes do not introduce new bugs or performance issues. ## Proposed solution Implement in-place writes for the B-Tree cache files. If the size of the data being written is the same or smaller than the previous data, the node should be written to the same location in the file. This can be achieved around lines #371 to #375 of Btree.php, which is where the serialized write occurs. ## Examples N/A ## Priority/Severity - [x] High (This will bring a huge increase in performance/productivity/usability/legislative cover) - [ ] Medium (This will bring a good increase in performance/productivity/usability) - [ ] Low (anything else e.g., trivial, minor improvements)

jamie commented

2021-03-10 10:44:06 +00:00

(Migrated from git.hazaar.io)

changed title from H3: B{-t-}ree in-place writes to H3: B{+-T+}ree in-place writes

changed title from **H3: B{-t-}ree in-place writes** to **H3: B{+-T+}ree in-place writes**

jamie commented

2021-03-10 22:27:15 +00:00

(Migrated from git.hazaar.io)

changed title from {-H3: -}B-Tree in-place writes to B-Tree in-place {+node +}writes

changed title from **{-H3: -}B-Tree in-place writes** to **B-Tree in-place {+node +}writes**

jamie commented

2024-06-13 14:16:26 +00:00