You may need to use the gpu_memory_limit and/or lora_on_cpu config options in order to avoid working outside of memory. If you continue to operate away from CUDA memory, it is possible to make an effort to merge in https://mayazixw647897.blogspothub.com/profile