Issues

• Page Cache Management

  FMInference/FlexLLMGen#142 · Filter by author BL-GS opened on Nov 27, 2024

• LP optimization model and constants

  FMInference/FlexLLMGen#140 · Filter by author dimanzt opened on Oct 10, 2024

• How to split execution of prefill and decode for Flexgen?

  FMInference/FlexLLMGen#138 · Filter by author sunchaesk opened on Jul 9, 2024

• File "setup.py" not found.

  FMInference/FlexLLMGen#137 · Filter by author Learn2006 opened on May 3, 2024
  3

• Killed Issue with flexgen when running python script

  FMInference/FlexLLMGen#136 · Filter by author foreverpiano opened on Apr 15, 2024
  2

• AttributeError: 'NoneType' object has no attribute 'stream_id'

  FMInference/FlexLLMGen#134 · Filter by author neomi-tenenbaum-huawei opened on Mar 26, 2024

• Error while split the model name

  FMInference/FlexLLMGen#133 · Filter by author neomi-tenenbaum-huawei opened on Mar 26, 2024

• Why the variable bls must be less than 20？

  FMInference/FlexLLMGen#132 · Filter by author LHQUer opened on Mar 25, 2024

• How do I match the results of profiling with the parameters of the cost model?

  FMInference/FlexLLMGen#131 · Filter by author xvanQ opened on Jan 31, 2024
  1

• Implement RESTful API of FlexGen

  FMInference/FlexLLMGen#130 · Filter by author Fyphen1223 opened on Jan 25, 2024

• "helm_run.py, line 303, in run_entry run_spec = run_specs[0] IndexError: list index out of range"

  FMInference/FlexLLMGen#129 · Filter by author hjk1231 opened on Jan 18, 2024
  1

• what is the helm version?

  FMInference/FlexLLMGen#128 · Filter by author oujieww opened on Jan 9, 2024
  1

• How to use the model that has already been downloaded？

  FMInference/FlexLLMGen#127 · Filter by author AntonioZC666 opened on Jan 7, 2024
  2

• Please do not abandon this project!

  FMInference/FlexLLMGen#126 · Filter by author oobabooga opened on Nov 27, 2023
  3

• 【PLS!】I want to know how to generate ray_bootstrap_config.yaml for my own cluster

  FMInference/FlexLLMGen#123 · Filter by author KylinC opened on Aug 30, 2023

• question about quantization

  FMInference/FlexLLMGen#119 · Filter by author xinhaoc opened on Jun 4, 2023

• how to install from source

  FMInference/FlexLLMGen#118 · Filter by author SeekPoint opened on May 28, 2023
  4

• Why is the CPU peak memory usage set to 0?

  FMInference/FlexLLMGen#117 · Filter by author KAIWEILIUCC opened on May 12, 2023

• AttributeError: 'OptLM' object has no attribute 'weight_home'

  FMInference/FlexLLMGen#116 · Filter by author pxc3113 opened on May 11, 2023
  2

• flexgen without GPU?

  FMInference/FlexLLMGen#115 · Filter by author AnatoliChe opened on May 4, 2023

• NotImplementedError on --percent 50 50 50 50 50 50

  FMInference/FlexLLMGen#114 · Filter by author SeekPoint opened on Apr 29, 2023

• Could flexgen be used for training?

  FMInference/FlexLLMGen#113 · Filter by author leiwen83 opened on Apr 25, 2023

• Support for MoE models (see Switch Tranformer, NLLB)

  FMInference/FlexLLMGen#109 · Filter by author fiqas opened on Apr 18, 2023

• Peak gpu memory use not scale linearly with the percentage of gpu usage of weight

  FMInference/FlexLLMGen#108 · Filter by author frankxyy opened on Apr 17, 2023

• When will the optimizer for determining offload strategy be released?

  FMInference/FlexLLMGen#107 · Filter by author frankxyy opened on Apr 17, 2023