Apache TVM v0.23.0

Introduction

The TVM community has worked since the last release to deliver the following new exciting improvements!

The main tags are below (bold text is with lots of progress): Relax (especial PyTorch frontend), TIR etc.

Please visit the full listing of commits for a complete view: v0.23.dev0...v0.23.0.rc0.

Community

None.

RFCs

None.

Adreno

• #18523 - [TEXTURE] Texture based lowering

Arith

• #18542 - Revert "Fix InternalError: Check failed: (eval_vec_) is false"
• #18536 - Fix InternalError: Check failed: (eval_vec_) is false

BugFix

• #18628 - [Fix] Fix typo in file header comment
• #18589 - [OpenCL] Guard QCOM perf hint behind USE_OPENCL_EXTN_QCOM to avoid undefined symbol on non-QCOM runtimes
• #18534 - Prevent segfault when instantiating abstract SearchStrategy

CI

• #18549 - Remove hardcoded user and repo values
• #18484 - Update file patterns for specific linting hooks
• #18470 - Enhance python linting scripts to support revision-based checks
• #18498 - Use glob for conda/build-environment.yaml in cache key
• #18495 - Update actions/cache to v4 in setup action
• #18457 - Fix crash when grep finds no matches
• #18448 - Update pre-commit configuration
• #18432 - Enable username checks in PR title and body
• #18430 - [TEST][CODEGEN] Fix the test scripts tries to tell numpy a dtype name that it cannot recognise
• #18419 - [TEST] Refactor: remove the deprecated warning message check from test cases

Docs

• #18545 - Improve static shape tuning parameter configuration (follow-up to commit c71aefc)
• #18539 - Fix e2e_opt_model tutorial for GPU deployment
• #18451 - Update the merge setting
• #18436 - Remove prebuilt package references and disable Colab button at tutorials
• #18413 - Update cross-compilation and RPC tutorial with modern PyTorch deployment workflow
• #18412 - Update tutorial for exporting and loading back Relax executables
• #18404 - Add tutorial for exporting and loading back Relax executables

Frontend

• #18435 - [ONNX] Fix operator Transpose: TVMError: PermuteDims expects the number of input axes to equal the ndim of the input tensor

LLVM

• #18586 - [Codegen] Avoid segfault when arith::GetVScaleValues returns empty vector

MetaSchedule

• #18547 - Fix tune_tir crash with ScheduleError in RewriteParallelVectorizeUnroll

Relax

• #18676 - Implement dynamic output trimming for NMS
• #18664 - Add FDataDependent operator attribute for LegalizeOps
• #18668 - [Onnx] Support Local Response Normalization (LRN)
• #18667 - Add native size operator
• #18675 - [LAYOUT] Support for dynamic layout specification
• #18652 - [ONNX] add support for unique optional outputs
• #18665 - Replace topi.take with relax.op.take
• #18663 - Fix wrong memory planning when only lower bound was provided
• #18666 - [Onnx][Resize] Handle non-4D input tensors
• #18658 - [Onnx][PReLU] Handle slope and axis argument with different slope shapes
• #18649 - Remove obsolete TODO comments
• #18642 - Add FRelaxInferLayout for gather_elements operator
• #18643 - Add FRelaxInferLayout for scatter_nd operator
• #18641 - [Op] Fixed incorrect output shape of Pool op when ceil_mode = true
• #18638 - Add FRelaxInferLayout for scatter_elements operator
• #18637 - Add FRelaxInferLayout for flip operator
• #18633 - Add FRelaxInferLayout and TMixedPrecisionPolicy for dynamic_strided_slice
• #18635 - [Onnx] Pass output_padding param in ConvTranspose
• #18632 - Move GetUsedVars to analysis module
• #18629 - Add FInferMixedPrecision and FRelaxInferLayout for conv transpose ops
• #18626 - [Op][PyTorch] Supported Median operator
• #18576 - Correct YaRN RoPE frequency scaling formula to align with the original paper
• #18615 - Add gpu-generic fallback for unrecognized GPU targets
• #18621 - Use weight shape instead of dim in Embedding.forward
• #18613 - Remove duplicated test case: test_if_branch_var_scope
• #18616 - Replaced call_pure_packed with tensor_to_shape operator
• #18593 - feat: Implement FRelaxInferLayout for tile operator
• #18618 - Add test case for op attributes in AST printer
• #18619 - [PyTorch] Fix PyTorch Dynamo frontend for Darwin compatibility
• #18575 - [ONNX] Add edge padding mode
• #18620 - Fix flaky test_conv2d gradient numeric test
• #18609 - Fix batch normalization computation logic
• #18574 - [Torch] AssertionError: Unsupported function types ['mean.default']
• #18591 - Chore: Fix the DeprecationWarning: invalid escape sequence \
• #18577 - Clean up scatter_elements unknown dtype handling
• #18579 - Add layout inference support for repeat operator
• #18583 - [Torch] Fixed issues related to sum op when without dim and keep dim
• #18554 - Enhance unique block name generation with numeric suffixes
• #18558 - Add edge padding mode
• #18559 - Add mod operator support
• #18544 - [PyTorch] Add support for Custom Ops for ExportedProgram frontend
• #18535 - [PyTorch] Add support for masked_select
• #18551 - [Frontend] Introduce ModuleDict
• #18550 - [PyTorch] Enhance scale_factor handling in interpolation
• #18553 - [PyTorch] Unify dtype used in conv2d tests
• #18548 - [PyTroch] Add NHWC layout support
• #18533 - [PyTorch] Fix index_put with broadcast indices
• #18521 - [PyTorch] Handle unknown output shapes for _sym_size_int
• #18532 - [PyTorch] Add support for bidirectional GRU
• #18530 - [PyTorch] Add boolean tensor support for max operation and corresponding test case
• #18524 - [PyTorch] Fix InternalError when converting scaled_dot_product_attention with 2D inputs
• #18527 - [PyTorch] Add support for non-persistent buffers in ExportedProgram frontend
• #18529 - [PyTorch] Add support for binary scalar operations in ExportedProgram frontend and corresponding tests
• #18522 - [PyTorch] Unify tests using shared tvm.testing.assert_allclose
• #18516 - [PyTorch] Add support for bidirectional LSTM
• #18499 - [PyTorch] Add support for sparse matrix multiplication
• #18518 - [PyTorch] Fix batch normalization training mode correctness
• #18517 - [PyTorch] Unify tests using shared verify_model
• #18506 - [PyTorch] Enhance data type handling in FX graph translator
• #18507 - [PyTorch] Support specifying decimals for _round
• #18500 - [PyTorch] Add support for antialiased bilinear upsampling
• #18489 - [PyTorch] Enhance handling of unbounded upper bound constraints
• #17599 - [PASS] Annotate Custom Scope layout pass for Adreno GPU
• #18497 - [PyTorch] Add binary operation dtype promotion following PyTorch rules in ExportedProgram frontend
• #18478 - Fix the squeeze operator to behave consistently with torch
• #18496 - [PyTorch] Add mul operator in ExportedProgram frontend
• #18494 - [PyTorch] Add negative slicing support in slice_scatter operation
• #18493 - [PyTorch] Add broadcast support for copy operation
• #18490 - [PyTorch] Add as_strided operator in ExportedProgram frontend
• #18487 - [PyTorch] Add count_include_pad support to avg_pool2d in PyTorch frontend
• #18488 - [PyTorch] Enhance index_put support for multi-dimensional indices
• #18486 - [PyTorch] Fix batch_norm.default args handling in ExportedProgram frontend
• #18483 - [PyTorch] Add support for grid_sample operator
• #18482 - [PyTorch] Add support for gumbel_softmax
• #18485 - [PyTorch] Add dynamic shape support to torch.ops.aten.sym_size.int in ExportedProgram frontend
• #18473 - [PyTorch] Add support for torch.ops.aten.sym_size.int in ExportedProgram frontend
• #18471 - [PyTorch] Enable run_ep_decomposition by default
• #18462 - [PyTorch] Add decomposed operator support for interpolate
• #18455 - Fix flaky test_conv2d_offload by increasing float32 tolerance
• #18463 - [PyTorch] Support advanced range constraints (multiplication)
• #18464 - [PyTorch] Enable decomposition in all tests
• #18461 - [PyTorch] Fix KeyError: dtype when converting PyTorch model with gradient checkpointing using torch.export
• #18452 - [PyTorch] Support advanced range constraints (addition)
• #18454 - [PyTorch]: Fix the sqrt operation requires float dtype but receives int64 in attention scaling
• #18459 - [PyTorch] Fix MultiheadAttention complie
• #18460 - [PyTorch] Add decomposed operator support for normalization
• #18458 - [PyTorch] Add decomposed operator support for Binary
• #18449 - [PyTorch] Add decomposed operator support for Pad
• #18447 - [PyTorch] Add lower bound support for range constraints
• #18446 - [PyTorch] Add decomposed operator support for MaxPool
• #18437 - [PyTorch] Add decomposed operator support for AdaptiveAvgPool
• #18433 - [PyTorch] Add decomposed operator support for Conv
• #18429 - [PyTorch] Support basic range constraints
• #18428 - [PyTorch] Add support for decomposed operators and fix IR of ops tests(8)
• #18427 - [PyTorch] Add support for decomposed operators and fix IR of ops tests(7)
• #18420 - [PyTorch] Add support for decomposed operators and fix IR of ops tests(6)
• #18417 - [PyTorch] Add support for decomposed operators and fix IR of ops tests(5)
• #18416 - [ONNX] Fix bug: Unsupported numpy or ml_dtypes dtype('O') when importing ONNX model using Relax frontend
• #18414 - [PyTorch] Add support for decomposed operators and fix IR of ops tests(4)
• #18410 - [PyTorch] Add support for decomposed operators and fix IR of ops tests(3)
• #18403 - [PyTorch] Add support for decomposed operators and fix IR of ops tests(2)
• #18402 - [PyTorch] Add support for decomposed operators and fix IR of ops tests(1)
• #18401 - [PyTorch] Enable decomposition for unary ops and refactor tests
• #18400 - [PyTorch] Add support for decomposed operators in extended unary ops tests
• #18399 - [PyTorch] Add run_ep_decomposition flag to control PyTorch decomposition

Runtime

• #18546 - [MatchShape] Type error: Cannot convert from type ' DLTensor* ' to ' ffi.Shape '

TIR

• #18639 - [Schedule] Fix type checker to support subscripted generics in Python 3.14+
• #18515 - [Schedule] FuseReductionEpilogue: Add Clipping pattern support
• #18556 - [Schedule] Fix bug on bfloat16 conversion
• #18528 - [Schedule] Fix mma tensorize error
• #18514 - Fix tir.LowerIntrin check failed additional_info.size() == new_size
• #18505 - Update function signatures for decompose_reduction
• #18479 - : Fix VerifyStream::Verify causes dereferencing an invalid pointer
• #18421 - Add step attribute to ForNode (Initial codes)
• #18418 - [Schedule] Add FuseReductionEpilogue primitive to fuse epilogue …
• #18466 - Fix Data Type Mismatch (int64 vs int32) in T.match_buffer when Working with Scalar Buffers in TIR

TVMScript

• #18504 - Add test for TIR macro block name suffix handling
• #18465 - Add block name suffix management for TIR macros

cuda & cutlass & tensorrt

• #18624 - [CUDA] Fix cuModuleUnload crash during interpreter shutdown
• #18604 - [CUDA][FFI] Extend kernel launch config to support Programmatic Dependent Launch and cuLaunchCooperativeKernel

web

• #18683 - Fix RPC argument parsing for new FFI string/bytes types
• #18686 - Fix incorrect FFI export name in runtime.ts
• #18480 - Bump web runtime version 0.23.0-dev1
• #18467 - Replace string with TVMFFIByteArray* to avoid memory issues
• #18450 - Fix progress reporting when loading from cache
• #18415 - Fix arrayDecodeStorage scope issue for q0f32 models
• #18385 - Upgrade web runtime to new FFI

Misc

• #18681 - [NVRTC] Add NVSHMEM support to NVRTC compilation path
• #18674 - fix: MSVC pragma
• #18654 - [FFI] bump to latest version
• #18656 - Put options before objects when compiling
• #18519 - [Compile] accelerate compilation speed using NVRTC
• #18582 - Fix ACOS precision issue for boundary values (x=±1.0)
• #18557 - [Attn] Fix calling FlashInfer attention plan function
• #18555 - Fix duplicate PresburgerSetNode registration when USE_MLIR=ON and MLIR >= 15.0
• #18525 - [Schedule] Fix LocalBuilder Check failed: (index_map_func.has_value()) is false
• #18511 - [Pass] Add DumpIR pass instrument to save IR snapshots
• #18512 - Remove unused TVMC configs
• #18509 - Fix compilation warnings
• #18492 - Fix BufferError when converting PyTorch models with sparse tensors
• #18469 - [Contrib] Update RandomFill to use StreamSync for CUDA synchronization
• #18453 - [DataType] Update to use explicit Bool Type Aligning with DLPack
• #18422 - Adjusted Longrope embedding function to match Huggingface Implementation
• #18426 - Support integer type input for log and log2
• #18411 - [FFI] Bump tvm-ffi to latest
• #18409 - Fixing database bug
• #18390 - Support integer types in TIR expression operators
• #18398 - fix the 8-bit vector loads/stores problem, which will solve the problem raised in the codegen test for cuda
• #18389 - Add VisitStmt_ method for AssertStmtNode and StringImmNode
• #18361 - [WebLLM] Replace int64s with int32s in WebGPU kernels
• #18384 - Fix crash when multiple PrimFunc objects are present in IRModule
• #18378 - [release][Dont Squash] Update version to 0.22.0 and 0.23.0.dev on main branch