outqueue.h File Reference

Output queue handling in multithreaded coding. More...

#include "common.h"

Go to the source code of this file.

Data Structures
struct	lzma_outbuf_s
struct	lzma_outq

Typedefs
typedef struct lzma_outbuf_s	lzma_outbuf
	Output buffer for a single thread.

Functions
uint64_t	lzma_outq_memusage (uint64_t buf_size_max, uint32_t threads)
	Calculate the memory usage of an output queue.
lzma_ret	lzma_outq_init (lzma_outq *outq, const lzma_allocator *allocator, uint32_t threads)
	Initialize an output queue.
void	lzma_outq_end (lzma_outq *outq, const lzma_allocator *allocator)
	Free the memory associated with the output queue.
void	lzma_outq_clear_cache (lzma_outq *outq, const lzma_allocator *allocator)
	Free all cached buffers that consume memory but aren't in use.
void	lzma_outq_clear_cache2 (lzma_outq *outq, const lzma_allocator *allocator, size_t keep_size)
	Like lzma_outq_clear_cache() but might keep one buffer.
lzma_ret	lzma_outq_prealloc_buf (lzma_outq *outq, const lzma_allocator *allocator, size_t size)
	Preallocate a new buffer into cache.
lzma_outbuf *	lzma_outq_get_buf (lzma_outq *outq, void *worker)
	Get a new buffer.
bool	lzma_outq_is_readable (const lzma_outq *outq)
	Test if there is data ready to be read.
lzma_ret	lzma_outq_read (lzma_outq *restrict outq, const lzma_allocator *restrict allocator, uint8_t *restrict out, size_t *restrict out_pos, size_t out_size, lzma_vli *restrict unpadded_size, lzma_vli *restrict uncompressed_size)
	Read finished data.
void	lzma_outq_enable_partial_output (lzma_outq *outq, void(*enable_partial_output)(void *worker))
	Enable partial output from a worker thread.

Detailed Description

Output queue handling in multithreaded coding.

Typedef Documentation

lzma_outbuf

typedef struct lzma_outbuf_s lzma_outbuf

Output buffer for a single thread.

Function Documentation

lzma_outq_clear_cache()

void lzma_outq_clear_cache ( lzma_outq * outq, const lzma_allocator * allocator )

extern

Free all cached buffers that consume memory but aren't in use.

lzma_outq_clear_cache2()

void lzma_outq_clear_cache2 ( lzma_outq * outq, const lzma_allocator * allocator, size_t keep_size )

extern

Like lzma_outq_clear_cache() but might keep one buffer.

One buffer is not freed if its size is equal to keep_size. This is useful if the caller knows that it will soon need a buffer of keep_size bytes. This way it won't be freed and immediately reallocated.

lzma_outq_enable_partial_output()

void lzma_outq_enable_partial_output ( lzma_outq * outq, void(* enable_partial_output )(void *worker) )

extern

Enable partial output from a worker thread.

If the buffer at the head of the output queue isn't finished, this will call enable_partial_output on the worker associated with that output buffer.

Note

This reads a lzma_outbuf.finished variable and thus calls to this function need to be protected with a mutex.

lzma_outq_end()

void lzma_outq_end ( lzma_outq * outq, const lzma_allocator * allocator )

extern

Free the memory associated with the output queue.

lzma_outq_get_buf()

lzma_outbuf * lzma_outq_get_buf ( lzma_outq * outq, void * worker )

extern

Get a new buffer.

lzma_outq_prealloc_buf() must be used to ensure that there is a buffer available before calling lzma_outq_get_buf().

lzma_outq_init()

lzma_ret lzma_outq_init ( lzma_outq * outq, const lzma_allocator * allocator, uint32_t threads )

extern

Initialize an output queue.

Parameters

outq	Pointer to an output queue. Before calling this function the first time, *outq should have been zeroed with memzero() so that this function knows that there are no previous allocations to free.
allocator	Pointer to allocator or NULL
threads	Number of buffers that may be in use concurrently. Note that more than this number of buffers may actually get allocated to improve performance when buffers finish out of order. The actual maximum number of allocated buffers is derived from the number of threads.

Returns

- LZMA_OK

• LZMA_MEM_ERROR

lzma_outq_is_readable()

bool lzma_outq_is_readable ( const lzma_outq * outq )

extern

Test if there is data ready to be read.

Call to this function must be protected with the same mutex that is used to protect lzma_outbuf.finished.

lzma_outq_memusage()

uint64_t lzma_outq_memusage ( uint64_t buf_size_max, uint32_t threads )

extern

Calculate the memory usage of an output queue.

Returns

Approximate memory usage in bytes or UINT64_MAX on error.

lzma_outq_prealloc_buf()

lzma_ret lzma_outq_prealloc_buf ( lzma_outq * outq, const lzma_allocator * allocator, size_t size )

extern

Preallocate a new buffer into cache.

Splitting the buffer allocation into a separate function makes it possible to ensure that way lzma_outq_get_buf() cannot fail. If the preallocated buffer isn't actually used (for example, some other error occurs), the caller has to do nothing as the buffer will be used later or cleared from the cache when not needed.

Returns

LZMA_OK on success, LZMA_MEM_ERROR if allocation fails

lzma_outq_read()

lzma_ret lzma_outq_read ( lzma_outq *restrict outq, const lzma_allocator *restrict allocator, uint8_t *restrict out, size_t *restrict out_pos, size_t out_size, lzma_vli *restrict unpadded_size, lzma_vli *restrict uncompressed_size )

extern

Read finished data.

Parameters

outq	Pointer to an output queue
out	Beginning of the output buffer
out_pos	The next byte will be written to out[*out_pos].
out_size	Size of the out buffer; the first byte into which no data is written to is out[out_size].
unpadded_size	Unpadded Size from the Block encoder
uncompressed_size	Uncompressed Size from the Block encoder

Returns

- LZMA: All OK. Either no data was available or the buffer being read didn't become empty yet.

• LZMA_STREAM_END: The buffer being read was finished. *unpadded_size and *uncompressed_size were set if they were not NULL.

Note

This reads lzma_outbuf.finished and .pos variables and thus calls to this function need to be protected with a mutex.