AppleInferenceManager

class qlip.inference.apple.AppleInferenceManager(*args, key=None, **kwargs)

Bases: InferenceManager

Apple inference manager.

Parameters

  • *args (tuple) – Arguments for parent class.

  • **kwargs (dict) – Keyword arguments for parent class.

backend

alias of AppleBackend

auto_setup()

Auto setup the model from the workspace.

classmethod from_compilemanager(cmanager, inference_config=None, original_device='meta')

Initialize inference manager from compile manager.

Parameters

  • cmanager (CompileManager) – Compile manager to initialize from.

  • inference_config (SessionConfig) – Configuration for inference session.

  • original_device (str) – Move original model to this device.

setup_model(*, inference_config=None, device='cuda', original_device='meta', adapter_type='auto')

Setup the model from the workspace.

Parameters

  • inference_config (SessionConfig) – Configuration for inference session.

  • device (str) – Device to use for inference.

  • original_device (str) – Move original model to this device.

  • adapter_type (str) – Type of adapter to use, by default ‘auto’. Possible values are: ‘auto’, ‘default’, ‘hf_adapter’, ‘hf_unet_adapter’.

setup_modules(*, modules=None, exclude_modules=None, inference_config=None, component=None, device='cuda', original_device='meta', adapter_type='auto')

Setup modules for inference.

Parameters

  • modules (Iterable[str]) – Names of modules to include.

  • exclude_modules (Iterable[str]) – Names of modules to exclude.

  • inference_config (SessionConfig) – Configuration for inference session.

  • component (Optional[str]) – Component name. For DiffusionPipeline from diffusers.

  • device (str) – Device to use for inference.

  • original_device (str) – Move original model to this device.

  • adapter_type (str) – Type of adapter to use, by default ‘auto’. Possible values are: ‘auto’, ‘default’, ‘hf_adapter’, ‘hf_unet_adapter’.