MDBF

MDBF Quantizer

MDBF dataclass

MDBF(name: str = None, num_layers: int = None, calc_quant_error: bool = False, bitpack_on_quantize: bool = False, include_layer_names: list[str] = None, exclude_layer_names: list[str] = (lambda: ['lm_head'])(), include_layer_keywords: list[str] = None, exclude_layer_keywords: list[str] = (lambda: ['per_layer_model_projection'])(), target_layer_types: tuple = (lambda: (Linear,))(), hessian_dtype: dtype = torch.float32, module_to_name: dict = dict(), results: dict = dict(), flag_calibration: bool = True, flag_hessian: bool = True, flag_xtx: bool = False, flag_nsamples: bool = True, flag_qep_supported: bool = True, target_bits: float = 1.0, l: int = DEFAULT_L, P: int = DEFAULT_P, svd_mode: str = 'svd', use_admm: bool = True, admm_outer_iters: int = 260, admm_inner_iters: int = 3, admm_reg: float = 0.03, admm_seed: Optional[int] = None, use_gradient_refine: bool = False, gradient_iters: int = 1000, gradient_lr: float = 0.01, activation_aware: bool = False, act_init: str = 'osvd', scale_bits: int = DEFAULT_SCALE_BITS, mlp_target_bits: Optional[float] = None, module_target_bits: Optional[dict[str, float]] = None)

Bases: Quantizer

MDBF quantizer.

Runs MDBF (Multi-Envelope Double Binary Factorization) quantization per layer.

Attributes:

Name	Type	Description
flag_calibration	bool	Calibration mode flag.
flag_hessian	bool	Hessian computation flag.
target_bits	float	Target BPW (e.g., 1.0).
l	int	Multi-scale (envelope) rank. l=1 collapses the envelope to rank one; (l, P) = (1, 1) is DBF and (1, 2) is LittleBit.
P	int	Number of passes (1 or 2).
svd_mode	str	SVD initialization mode ("svd" or "svd_llm").
use_admm	bool	Whether to use ADMM optimization.
admm_outer_iters	int	ADMM outer iterations.
admm_inner_iters	int	ADMM inner iterations.
admm_reg	float	ADMM regularization coefficient.
admm_seed	Optional[int]	Random seed (int in [0, MAX_SEED]) for the randomized SVD initialization inside the ADMM MDBF projection. None uses the global RNG, so results depend on the ambient RNG state; setting an integer makes the ADMM phase reproducible regardless of that state.
use_gradient_refine	bool	Whether to use gradient refinement.
gradient_iters	int	Gradient refinement iterations.
gradient_lr	float	Gradient refinement learning rate.
activation_aware	bool	Whether to use activation-aware mode (P=1 only).
act_init	str	Activation initialization mode.
scale_bits	int	Bit-width used to account for the FP16 amplitude scales when sizing the rank and reporting BPW (accounting only; does not change the stored dtype). 16 = FP16, 0 = binary-only.
mlp_target_bits	float	BPW override for MLP layers.
module_target_bits	dict	Per-layer BPW override.

Methods:

Name	Description
quantize_layer	Quantizes a given layer using MDBF.

resolve_bits staticmethod

resolve_bits(layer_name: Optional[str], default_bits: float, mlp_bits: Optional[float] = None, module_bits: Optional[dict[str, float]] = None) -> float

Resolve bit-width from overrides (module > mlp > default).

Used by the quantizer and by config loader. If layer_name is None, returns default_bits.

validate_params

validate_params()

Validate MDBF parameters once in setup().

Validated ranges

target_bits: float > 0 l: int >= 1 P: int in {1, 2} admm_outer_iters: int >= 1 (when use_admm=True) admm_reg: float >= 0 admm_seed: int in [0, MAX_SEED] or None gradient_iters: int >= 1 (when use_gradient_refine=True) gradient_lr: float > 0

quantize_layer

quantize_layer(module: Module, input=None, hessian: Tensor = None, nsamples: Optional[int] = None) -> MDBFResult

Quantize the layer using MDBF.

Parameters:

Name	Type	Description	Default
module	Module	The layer module.	required
input	tuple or Tensor	The input to the layer (activations).	None
hessian	Tensor	The Hessian matrix.	None
nsamples	int	Number of tokens used to compute the Hessian.	None

Returns:

Name	Type	Description
MDBFResult	MDBFResult	MDBF quantization result.

get_quant_config

get_quant_config() -> dict

Return quantization_config dict for save_quantized_model.

All values record the requested quantizer configuration. finalize_quant_config_for_save() additionally records "actual_activation_aware" (what was actually used after any per-layer fallback).

create_inference_layer

create_inference_layer(result, linear_module, **kwargs)

Build MultipathMDBFLinear from MDBFResult.

MDBFResult

MDBFResult dataclass

MDBFResult(dequantized_weight: Tensor = None, quantization_time: float = None, output_squared_error: float = None, mean_output_squared_error: float = None, weight_squared_error: float = None, mean_weight_squared_error: float = None, relative_output_squared_error: float = None, relative_weight_squared_error: float = None, target_bits: float = None, l: int = None, P: int = None, svd_mode: str = None, use_admm: bool = None, admm_outer_iters: int = None, admm_inner_iters: int = None, admm_reg: float = None, admm_seed: Optional[int] = None, use_gradient_refine: bool = None, gradient_iters: int = None, gradient_lr: float = None, activation_aware: bool = None, act_init: str = None, scale_bits: int = None, actual_activation_aware: Optional[bool] = None, actual_bpw: float = None, r: int = None, is_mdbf_quantized: Optional[bool] = None, mdbf_A_sign: list = list(), mdbf_B_sign: list = list(), mdbf_A_amp: list = list(), mdbf_B_amp: list = list(), mdbf_Q_U_amp: list = list(), mdbf_Q_V_amp: list = list())

Bases: QuantizationResult

MDBF quantization result.

Attributes:

Name	Type	Description
target_bits	float	Target BPW (e.g., 1.0).
l	int	Multi-scale rank.
P	int	Number of passes.
svd_mode	str	SVD initialization mode.
use_admm	bool	Whether ADMM optimization was used.
admm_outer_iters	int	ADMM outer iterations.
admm_inner_iters	int	ADMM inner iterations.
admm_reg	float	ADMM regularization coefficient.
admm_seed	Optional[int]	Random seed used for the ADMM MDBF projection (None = global RNG).
use_gradient_refine	bool	Whether gradient refinement was used.
gradient_iters	int	Gradient refinement iterations.
gradient_lr	float	Gradient refinement learning rate.
activation_aware	bool	Requested activation-aware setting (as configured on the quantizer). See actual_activation_aware for what was used.
act_init	str	Activation initialization mode.
actual_activation_aware	bool	Whether activation-aware mode was actually used. run_mdbf falls back to non-aware mode when P != 1 or when no Hessian is supplied, so this may be False even if activation_aware is True. None means unknown (run_mdbf did not report it).
scale_bits	int	Bit-width used to account for the FP16 amplitude scales when sizing the rank and reporting BPW (accounting only; does not change the stored dtype). 16 = FP16, 0 = binary-only.
actual_bpw	float	Achieved BPW.
r	int	Rank used.
is_mdbf_quantized	bool	Whether MDBF quantization was applied.
mdbf_A_sign	list	Sign matrices S_A per pass [(n, r)] × P.
mdbf_B_sign	list	Sign matrices S_B per pass [(r, m)] × P.
mdbf_A_amp	list	Row-scale matrices per pass [(n, l)] × P.
mdbf_B_amp	list	Column-scale matrices per pass [(m, l)] × P.
mdbf_Q_U_amp	list	Latent row-scale matrices per pass [(r, l)] × P.
mdbf_Q_V_amp	list	Latent column-scale matrices per pass [(r, l)] × P.

get_MDBF_params_list

get_MDBF_params_list() -> list[MDBFParams]

Validate stored per-path tensors and convert them to MDBFParams objects.

compute_dequantized_weight

compute_dequantized_weight(device=None) -> torch.Tensor

Compute dequantized weight from quantized parameters.

Reconstructs W ≈ Σ_p F^(p) @ G^(p) from the stored per-pass tensors.

Parameters:

Name	Type	Description	Default
device	str or device	Device to compute on.	None

Returns:

Type	Description
Tensor	Dequantized weight tensor (FP16, CPU).