Skip to content

vllm.core.block.utils

Block manager utils.

check_no_caching_or_swa_for_blockmgr_encdec 

check_no_caching_or_swa_for_blockmgr_encdec(
    block_mgr, seq_group: SequenceGroup
) -> None

Enforce that prefix caching & sliding-window attention (SWA) are currently unsupported specifically for encoder/decoder models.

Raises NotImplementedError if unsupported scenario is detected.

Arguments:

  • block_mgr: BlockSpaceManager instance
  • seq_group: SequenceGroup passed to block_mgr
Source code in vllm/core/block/utils.py 
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
def check_no_caching_or_swa_for_blockmgr_encdec(
        block_mgr, seq_group: SequenceGroup) -> None:
    '''
    Enforce that prefix caching & sliding-window attention (SWA)
    are currently unsupported *specifically* for encoder/decoder models.

    Raises NotImplementedError if unsupported scenario is detected.

    Arguments:

    * block_mgr: BlockSpaceManager instance
    * seq_group: SequenceGroup passed to block_mgr
    '''

    if seq_group.is_encoder_decoder():
        if block_mgr.max_block_sliding_window is not None:
            raise NotImplementedError(STR_NOT_IMPL_ENC_DEC_SWA)

        if block_mgr.enable_caching:
            raise NotImplementedError(STR_NOT_IMPL_ENC_DEC_PREFIX_CACHE)