See commit 47a2def4533a2807e48954abd50b32ecb1aaf29a and the next two
following it.
Thank you Zoltan for pointing out this commit, I agree that this seems to be
the trigger for the issues I'm seeing. Unfortunately the common CI host size
is 1-2G. For example on Ubuntu Autopkgtests 1.5G.
Those of them running guests do so in 0.5-1G size in TCG mode
(as they often can't rely on having KVM available).
The 1G TB buffer + 0.5G actual guest size + lack of dynamic downsizing
on memory pressure (never existed) makes these systems go OOM-Killing
the qemu process.
The patches indicated that the TB flushes on a full guest boot are a
good indicator of the TB size efficiency. From my old checks I had:
- Qemu 4.2 512M guest with 32M default overwritten by ram-size/4
TB flush count 14, 14, 16
- Qemu 5.0 512M guest with 1G default
TB flush count 1, 1, 1
I agree that ram/4 seems odd, especially on huge guests that is a lot
potentially wasted. And most environments have a bit of breathing
room 1G is too big in small host systems and the common CI system falls
into this category. So I tuned it down to 256M for a test.
- Qemu 4.2 512M guest with tb-size 256M
TB flush count 5, 5, 5
- Qemu 5.0 512M guest with tb-size 256M
TB flush count 5, 5, 5
- Qemu 5.0 512M guest with 256M default in code
TB flush count 5, 5, 5
So performance wise the results are as much in-between as you'd think from a
TB size in between. And the memory consumption which (for me) is the actual
current issue to fix would be back in line again as expected.
So on one hand I'm suggesting something like:
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -944,7 +944,7 @@ static void page_lock_pair(PageDesc **re
* Users running large scale system emulation may want to tweak their
* runtime setup via the tb-size control on the command line.
*/
-#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (1 * GiB)
+#define DEFAULT_CODE_GEN_BUFFER_SIZE_1 (256 * MiB)
#endif
#endif
OTOH I understand someone else might want to get the more speedy 1G
especially for large guests. If someone used to run a 4G guest in TCG the
TB Size was 1G all along.
How about picking the smaller of (1G || ram-size/4) as default?
This might then look like:
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -956,7 +956,12 @@ static inline size_t size_code_gen_buffe
{
/* Size the buffer. */
if (tb_size == 0) {
- tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
+ unsigned long max_default = (unsigned long)(ram_size / 4);
+ if (max_default < DEFAULT_CODE_GEN_BUFFER_SIZE) {
+ tb_size = max_default;
+ } else {
+ tb_size = DEFAULT_CODE_GEN_BUFFER_SIZE;
+ }
}
if (tb_size < MIN_CODE_GEN_BUFFER_SIZE) {
tb_size = MIN_CODE_GEN_BUFFER_SIZE;
This is a bit more tricky than it seems as ram_sizes is no more
present in that context but it is enough to discuss it.
That should serve all cases - small and large - better as a pure
static default of 1G or always ram/4?
P.S. I added Alex being the Author of the offending patch and Richard/Paolo
for being listed in the Maintainers file for TCG.
--
Christian Ehrhardt
Staff Engineer, Ubuntu Server
Canonical Ltd