Re: Should diskfs_file_update be called in every io_write?

[Sergio López]

2011/9/13 Thomas Bushnell, BSG <tb@becket.net>:
> I think this could also create extra churn, by scheduling a lot of disk
> writes when a series of writes are done all at once to a single file.

This is mitigated due to competition for the pager's lock between
data_requests and data_returns.

> Traditionally, this is addressed by the every-30-seconds update task; there
> is no expectation of an indefinitely postponed write.

Synchronizing only at a fixed interval (5 seconds is the default for
current libdiskfs), when writing a large file, induces the arrival of
a lot of data_return messages in a pretty short amount of time. This
is even worse when memory is scarce and GNU Mach starts flushing out
dirty pages.

> However, much could be improved in this area. Ideally, writes would happen
> at a smoothly increasing rate as page pressure increased; and ideally, the
> kernel would be correctly detected sequential access patterns and paging out
> finished pages speedily on its own.

I agree with you in this regard, but I'm not sure if should be Mach
the one who throttles page outs to the servers. Mach doesn't know how
fast the pagers are going to process each data_return, so it ends
flushing out a bunch of pages and waiting an arbitrary amount of time,
hoping servers will catch up.

I think pagers should be the ones in control of the page outs, by
calling m_o_lock_request at a reasonable rate. Also, when Mach's vm
pageout daemon is awaken, it should always try to free external clean
pages first (currently it doesn't even keep track of which ones are
external), and if the desired number of pages can't be freed because
most of them are dirty, a soft blockade (raised for memory objects
with certain attributes) should be imposed and all pagers notified,
perhaps by the use of an special exception raised at their page faults
that would generate a data_request. When the target is met, the
blockade is raised and all pagers can behave normally again.

Ideally, this should be done on a per-task basis, to prevent a rogue
translator from being able to compromise the entire system, but even
if it's implemented system-wide, an offender could be heuristically
found and killed (like the OOM Killer from Linux).