








                      Disk Quotas in NetBSD


                        David A. Holland
          Based upon an earlier version by Robert Elz.


                            ABSTRACT

          In  most computing environments, disk space is not
     infinite.  The disk quota system provides  a  mechanism
     to  control usage of disk space on an individual basis.

          Quotas may be set for each  individual  user,  and
     for  groups.   Quotas  may be set (or not set) indepen-
     dently on each mounted file  system.   Quotas  are  not
     supported  on  all  file system types, but do work over
     NFS.

          The quota system will warn users when they  exceed
     their  allotted  limit,  but allow some extra space for
     current work.  Remaining over quota for long periods of
     time  will  eventually  cause a fatal over-quota condi-
     tion.

          File system independent access to quotas  is  pro-
     vided  via  a library, libquota(3), shared by the quota
     tools and available to third-party software.  Dump  and
     restore  tools are provided for backing up quota infor-
     mation.


1.  Users' view of disk quotas

     To an ordinary user, having a disk quota is almost the  same
as  having a smaller disk.  When the amount of space available is
consumed, further attempts to use more will fail.

     To make this less painful, the quota  system  actually  pro-
vides  two  limits  for  every  resource: a hard limit and a soft
limit.  The hard limit may not be exceeded.  The soft  limit  may
be exceeded temporarily: when the soft limit is first exceeded, a
timer is initialized.  If usage is reduced below the soft  limit,
the  timer  is  cleared.   Otherwise, when the timer expires, the
soft limit becomes the hard limit: until usage is  reduced  below
the  soft  limit,  further use of space is denied.  The length of
time allowed is called the grace period and  is  configurable  by
the system administrator.

     Quotas may be applied to both users and groups.  When both a
user quota and a group quota apply, the most restrictive is used.









SMM:4-2                                     Disk Quotas in NetBSD


     The basic purpose of quotas is to restrict the usage of disk
space.  This is measured in  blocks.   (For  historical  reasons,
quotas are counted in terms of 512-octet blocks, even though most
file systems  use  larger  blocks  internally.)   Quotas  can  be
imposed  on  other resource types too.  Generally there will also
be a quota on the number of files that may be owned.  This  func-
tionality  is supported by nearly all file system types that sup-
port quotas at all.  Some file system  types  may  support  addi-
tional quotas on other file system resources.

     The quota(1) command provides information on the disk quotas
applying to a user or group.  For each resource  type,  and  each
file  system,  it  prints  the  current  usage,  the  soft  limit
(`quota'), the hard limit (`limit'), and the expiration  time  of
the  current  grace  period, if any.  User quotas are reported by
default; the -g flag allows retrieving group quotas.

     The soft limit is the usage level that the user is  expected
to  remain  below.   As  described  above,  the soft limit can be
exceeded temporarily; doing so generates  a  warning.   The  hard
limit  cannot be exceeded.  When the usage exceeds the applicable
limit, further requests for space (or attempts to create a  file,
or  allocate  other  resources)  will fail with the EDQUOT error.
The first time this occurs, a message  will  be  written  to  the
user's  terminal.   To  avoid  flooding,  the message will not be
repeated until after the usage is lowered again.

1.1.  Surviving when quota limit is reached

     In most cases, the only way to recover  from  an  over-quota
condition  is  to  stop whatever activity was in progress, remove
sufficient files to bring the limit back below quota,  and  retry
the failed program.

     Be  careful  not  to  exit  applications whose document save
operations have or may have failed because of the over-quota con-
ditions.   This  can  often  lead to losing data: the prior saved
version of the document may have been  replaced  with  an  empty,
partial, or invalid half-saved version.  If it is not possible to
open an additional terminal or other tools  to  do  the  clean-up
work,  use  job control to suspend these applications rather than
exiting.  It may also be possible to save documents to a  differ-
ent  file  system  (perhaps in /tmp) and then move them back into
place after cleaning up.

2.  Back-end quota implementations

     In NetBSD there are three categories  of  quota  implementa-
tions.   First  are file systems with native internal support for
quotas.  In these, the quota data is stored internally within the
file system, maintained by the file system's own consistency man-
agement and fsck(8) utility, and active whenever the file  system
volume is mounted.  The mechanism for setting up quotas on a par-
ticular volume is file-system-dependent  but  generally  involves









Disk Quotas in NetBSD                                     SMM:4-3


arguments  to  newfs(8)  and/or tunefs(8).  The ``quota2'' quotas
for FFS that appeared in NetBSD 6.0 are an example of this  type.

     The second category is NFS.  In NFS, quotas are handled by a
separate SunRPC protocol not directly connected to the NFS mount.
The  quota utilities speak this protocol when invoked on NFS vol-
umes.  On an  NFS  server,  the  rpc.rquotad(8)  daemon  must  be
enabled to serve this protocol.  See below for further details.

     The third category is instances of the historical quota sys-
tem, which was  developed  in  the  1980s  and  remained  largely
unchanged  until  being replaced during the development of NetBSD
6.0.  In the historical quota system, quotas are stored in exter-
nal  (user-visible)  files.   These appear by default in the root
directory of the volume they apply to, but  may  be  placed  any-
where,  including  sometimes on another file system.  While file-
system-level support is required, and enforcement is done in  the
file  system,  much of the work involved in maintaining the quota
system is external to the file system.  The program quotacheck(8)
is  run  at boot time as a form of fsck(8) for the quota informa-
tion.  Once this is done, the program quotaon(8) is used to acti-
vate  the  quotas.  The program quotaoff(8) is used to deactivate
the quotas during system shutdown.  This is normally done  before
unmounting;  if  not,  unmounting  will do it implicitly.  If the
file system volume is modified after quotaoff is run,  quotacheck
must  be  run before quotaon is run again, or the quota data will
become  corrupted.   Setting  up  the  historical  quota   system
requires  several  steps;  see below.  The original or ``quota1''
quotas for FFS are an example of this type of  quota  implementa-
tion.

     The  libquota(3)  library provides a common interface to all
three implementation types,  so  the  same  quota  administration
tools  can  be  used  with all of them.  Not all functionality is
available in all cases, however.  For  example,  NFS  quotas  are
read-only.   To adjust quotas on an NFS volume, one must log into
its file server.

3.  Setting up quotas

     To set up quotas of any type requires several steps.   These
are  described  in  detail,  for each file system type supporting
quotas, in the following sections.

     The first step in all cases, however, is to decide what file
systems  need  to have quotas.  Typically, only file systems that
house users' home directories, or other  user  files,  will  need
quotas.   Do not forget the mail spool.  It may also prove useful
to also include /usr.  If possible,  /tmp  should  preferably  be
free of quotas.

     It  is  also  necessary  to  decide what the quotas will be,
keeping in mind the available space and the legitimate  needs  of
the user population.









SMM:4-4                                     Disk Quotas in NetBSD


     Note  that because the quota system tracks per-user and per-
group disk usage, it can be useful in  some  contexts  to  enable
quotas  purely to provide that information.  It is perfectly rea-
sonable in this case to set quotas that are the same size as,  or
larger than, the space available on the volume.

3.1.  Setting up new (``quota2'') FFS quotas

     First,  make  sure  your  kernel includes FFS and FFS quota2
support.  These are present if the lines

     file-system  FFS
     options      QUOTA2

(respectively) are  present  in  the  kernel  configuration.   In
NetBSD  6.0 and higher the GENERIC kernels should all include FFS
and FFS quota2 support.  If FFS is compiled as a loadable module,
the module will always include quota2 support.

     Then,  when  creating  the volume, use the -q user and/or -q
group options to newfs(8) as desired.  This will create and  ini-
tialize  the on-disk data structures for user and/or group quotas
respectively.  These can also be created using tunefs(8).

     There is no need to set mount options in fstab(5).  In  par-
ticular,  do not set the userquota or groupquota options as these
are used for controlling the historic quota system only.   Trying
to  use  both quota systems at once will fail in possibly obscure
ways.

3.2.  Setting up quotas on an NFS client

     There is nothing special  that  needs  to  be  done  on  NFS
clients.  Enforcement (and configuration) of quotas occurs on the
NFS server.  The libquota(3) library recognizes  NFS  mounts  and
automatically uses the rquotad protocol to handle them.

3.3.  Setting up quotas on an NFS server

     To  set  up  quotas on an NFS server, first set up quotas on
the file systems that will be served via NFS.   Then  enable  the
rquotad(8)  service  in  /etc/inetd.conf(5).   Restart (or start)
inetd(8) if necessary.  Make sure

     inetd=YES

is present in /etc/rc.conf.  This is the default and  should  not
need to be changed explicitly.

3.4.  Setting up historical (``quota1'') FFS quotas

     First,  make  sure  your  kernel includes FFS and FFS quota1
support.  These are present if the lines










Disk Quotas in NetBSD                                     SMM:4-5


     file-system  FFS
     options      QUOTA

(respectively) are present in the kernel configuration.   In  all
NetBSD  versions  the  GENERIC kernels should include FFS and FFS
quota1 support.  If FFS is compiled as  a  loadable  module,  the
module will always include quota1 support.

     Note that it is possible that the historic quota system will
be removed entirely at some point in the  future.   This  is  not
presently intended, however.

     No  special options are required when creating the file sys-
tem.  Instead, add the userquota and/or groupquota options to the
file  system's entry in fstab(5).  The file system must be listed
in fstab(5) for the historical quota  system  to  work.   To  use
quota    files   other   than   the   default,   use   the   form
userquota=/path/to/file   and/or   groupquota=/path/to/file    as
desired.   The  default  files  are  quota.user  and  quota.group
respectively.  Create empty quota files with touch(1).

     If the file system is not brand new, now  run  quotacheck(8)
on  it.  (The file system must be mounted for this step.  Be sure
nothing else is writing to the file system during this time.)

     Now run quotaon(8) on the file system.

     You must also make sure that the setting

     quota=YES

is present in /etc/rc.conf.  This is the default and  should  not
need to be changed explicitly.  This setting causes quotacheck(8)
and quotaon(8) to be run at system boot time.

3.5.  Setting up historical quotas on other file system types

     In theory, the historical quota system can also be  used  on
LFS  and ext2fs file systems.  The procedure for this is the same
as for FFS.

     There is definitely at least some code  present  to  support
quotas  on  LFS;  however, as of this writing it is believed that
quotas do not actually work with either LFS or ext2fs.

4.  Quota administration

     After the quota system has been set up, and also on  a  con-
tinuing basis as users come and go and their requirements change,
the actual limit values need to be configured.  This is done with
the edquota(8) command.

     In  cases  where  large classes of users are to be given the
same quota settings, the -p option of edquota(8) can be  used  to









SMM:4-6                                     Disk Quotas in NetBSD


streamline the process.

     There is also a default entry.  Users (or groups) whose quo-
tas have not been set explicitly are  subjected  to  the  default
quotas instead.

     The  quota(1)  command  can be used to inspect an individual
user or group's quotas across all file systems.  The  repquota(8)
command can be used to retrieve quotas across all users or groups
on a per-file-system basis.

4.1.  Administrative considerations for native quotas

     The quotacheck(8), quotaon(8), and quotaoff(8) programs can-
not be used with native quotas.  They will fail if run.

     To  avoid  confusion  be  sure  not  to use the userquota or
groupquota options in fstab(5) on a file system with native  quo-
tas.   Due to implementation quirks of the historic quota system,
using these options will not necessarily cause  errors  at  mount
time, but may confuse quotacheck(8) and/or quotaon(8) and produce
bizarre results.

     Native quotas do not require quotacheck(8) and thus  can  be
much  faster  at boot time.  In particular, the native FFS quotas
used in conjunction with WAPBL journaling  are  themselves  jour-
naled and require only a journal replay after a crash.

     Note however that native FFS quotas are not backward-compat-
ible to older NetBSD installations.  As of this writing they  are
also not understood by FreeBSD's FFS implementation.

     There is currently no way to temporarily suspend enforcement
of native quotas.

4.2.  Administrative considerations for NFS quotas

     Most quota administration should (must) be done on  the  NFS
server  by  working  with  the volumes being served.  The rquotad
protocol is very limited and supports only the most basic  opera-
tions.  Notably, repquota(8) does not work on NFS volumes.

4.3.  Administrative considerations for historic quotas

     The historic quota system does not support all of the possi-
ble functionality.  There is no separate default entry,  and  the
grace  period  cannot  be  configured  individually; instead, one
grace period is configured for all users.

     In the historic quota system  the  default  values  and  the
global  grace  period  are  stored in the quota entry for UID (or
GID) 0.  This scheme partly shows through to the quota tools.  It
is  also  possible  to  attempt to establish quota configurations
that  cannot  be  represented.   These  will  fail.    edquota(8)









Disk Quotas in NetBSD                                     SMM:4-7


attempts  to detect these before submitting changes to the kernel
in order to offer a cogent error message;  however,  it  may  not
always succeed.

     Before  quotaon(8)  is  run,  the  quota  information is not
accessible to the kernel.  The libquota(3) library  detects  this
case  and  falls  back to direct access to the quota files.  This
should be fully transparent but it is possible that glitches  may
arise.

     It  is  possible to temporarily disable quota enforcement by
using quotaoff(8).  However, this also disables  usage  tracking.
Consequently,  if this is done while the system is live, it is in
general necessary to  run  quotacheck(8)  to  correct  the  usage
information  before running quotaon(8) again, and the file system
must be idle between the time quotacheck is started and the  time
quotaon  completes.  Otherwise the usage information in the quota
files will go out of sync with the file system.  This can lead to
improper behavior later on, and in some cases may cause panics.

     The  historical  quota system is 32-bit and thus cannot cope
with quotas or usage amounts that  cannot  be  represented  in  a
32-bit value.  Use native quotas on large volumes.

5.  Backups

     While  the  disk  usage information in the quota data can be
reconstructed by scanning the file  system  (this  is  what  quo-
tacheck  does),  the configured quotas themselves are system con-
figuration that should in general be backed up.  With the histor-
ical  quota  system,  the  quota information is stored in regular
files and is backed up in the normal way like other files.   How-
ever,  with native quotas, the quota information is hidden inside
the file system and additional steps are necessary to back it up.

     The  quotadump(8)  and quotarestore(8) programs are provided
for this purpose.

     The quotadump(8) program is the same as repquota(8) with the
-x option.  It produces a complete tabular dump of the quota set-
tings on the selected file system.  This dump file can  be  saved
on backup media.

     The  quotarestore(8)  reads  a dump file produced by quotad-
ump(8) and enters  the  configured  quota  information  into  the
selected  file system.  It can be used to restore from backup, or
to migrate quotas from one volume to another.

6.  Migrating to the new native FFS quota implementation

     The procedure for migrating from the historical (``quota1'')
FFS quotas to the native (``quota2'') FFS quotas is as follows.











SMM:4-8                                     Disk Quotas in NetBSD


     First,  make  sure the volume being migrated is fully backed
up.  This is important in case something goes wrong.

     Now, drop to single user  mode.   Then,  dump  the  existing
quota  information  with  quotadump(8).  Save the dump file some-
place safe, i.e. not in /tmp, in case  it  becomes  necessary  to
reboot.

     Unmount   the   volume.   Edit  /etc/fstab  and  remove  the
userquota and/or groupquota options.  Leave the old  quota  files
in place for now; they will do no harm.

     Use  tunefs(8)  to add native quotas to the file system.  If
the volume is very  old  you  might  first  need  to  update  the
superblock.

     Mount the file system.  Use quotarestore(8) to load the dump
file into the new quota system.

     Confirm using repquota(8) and/or quota(1) and/or by explicit
testing that the quotas have been loaded and the new quota system
is behaving as intended.  If paranoid, reboot and test again as a
precaution.

     Once  you  are fully satisfied, preferably after a few days'
usage, delete or archive the old quota files and  the  dump  file
used for transition.

     Remember to set up backup procedures for the quota data.

7.   Summary of changes from the historic to new/native FFS quota
implementations

     Quotas are set up with newfs or tunefs, rather than by edit-
ing fstab and running quotacheck.

     The  quotacheck,  quotaon,  and quotaoff tools are not used.
Repair is done with fsck instead.

     The quotas are integrated with  WAPBL  journaling,  allowing
fast crash recovery.

     The userquota and groupquota mount options are not used.

     The grace period is per-user instead of global.

     The defaults do not overlap with the id 0 quota entry.

8.  Some implementation details for the historic quotas

     The data in the quota files is an array of dquot structures,
indexed by id (UID or GID).  There is an entry for  every  id  on
the system, whether or not a quota is configured for that id.  If
the id space is sparse, then the  file  may  have  holes  in  it.









Disk Quotas in NetBSD                                     SMM:4-9


Copying  the files will fill in the holes, so it is best to avoid
this.

     The userquota and groupquota options are actually ignored by
mount(8).   They are instead found at run time by the quota tools
(actually by libquota); this is why file systems  using  historic
quotas must be listed in /etc/fstab.

     The  kernel  is  informed of the existence and identities of
the quota files by the  quotaon(8)  utility.   Until  this  point
(e.g. while quotacheck is running) the kernel does not know which
volumes are supposed to have quotas and which are not.

     When quotas are  turned  on,  the  kernel  reads  the  quota
entries  for  each user (or group) currently active, and then the
quota entries needed for  any  files  open  owned  by  users  (or
groups)  who are not currently active.  Each subsequent open of a
file on the file system will be accompanied by a pairing with its
quota  information.   In  most  cases  this  information  will be
retained in core, either because the user who owns  the  file  is
running  some  process, because other files are open owned by the
same user, or because some file (perhaps this one)  was  recently
accessed.   In memory, the quota information is kept hashed by id
and file system,  and  retained  in  an  LRU  chain  so  recently
released  data  can be easily reclaimed.  Information about those
users whose last process has recently terminated is also retained
in this way.

     Each  time a block is accessed or released, and each time an
inode is allocated or freed, the quota system gets told about it,
and in the case of allocations, gets the opportunity to object.

9.  History and acknowledgments

     The  historic  quota  system  was  loosely based upon a very
early scheme implemented at the University of  New  South  Wales,
and  Sydney University in the mid 70's. That system implemented a
single combined limit for both files and blocks on all file  sys-
tems.

     A  later  system  was  implemented at the University of Mel-
bourne by Robert Elz, but was not kept  highly  accurately.   For
example, chown did not affect quotas, nor did I/O to a file owned
by a different user.

     The historic quota system was put into place in January 1982
at Melbourne.

     For  the  4.2BSD release, much work was done to clean up and
sanely incorporate the quota code by Sam Leffler and  Kirk  McKu-
sick at The University of California at Berkeley.

     The  historic  quota  system  has remained in use (with only
minor modifications) through many BSD releases and versions  from









SMM:4-10                                    Disk Quotas in NetBSD


then right up until 2011, and remains available.

     In  2011,  Manuel  Bouyer implemented a new quota system for
FFS with the properties described above (in-FS, journaled,  etc.)
and  reworked  the  kernel  interface.   That interface was later
withdrawn.  In 2012 David A. Holland implemented a simpler kernel
interface  and  more comprehensive quota library and reworked the
quota tools to be file-system-independent.  This  material  first
appeared in NetBSD 6.0.



















































