Squid коды ошибок

The logs are a valuable source of information about Squid workloads and
performance. The logs record not only access information, but also
system configuration errors and resource consumption (e.g. memory, disk
space). There are several log file maintained by Squid. Some have to be
explicitly activated during compile time, others can safely be
deactivated during run-time.

There are a few basic points common to all log files. The time stamps
logged into the log files are usually UTC seconds unless stated
otherwise. The initial time stamp usually contains a millisecond
extension.

🔗 cache.log

The cache.log file contains the debug and error messages that Squid
generates. If you start your Squid using the -s command line option, a
copy of certain messages will go into your syslog facilities. It is a
matter of personal preferences to use a separate file for the squid log
data.

From the area of automatic log file analysis, the cache.log file does
not have much to offer. You will usually look into this file for
automated error reports, when programming Squid, testing new features,
or searching for reasons of a perceived misbehavior, etc.

🔗 Squid Error Messages

Error messages come in several forms. Debug traces are not logged at
level 0 or level 1. These levels are reserved for important and critical
administrative messages.

  • FATAL messages indicate a problem which has killed the Squid
    process. Affecting all current client traffic being supplied by that
    Squid instance.
    If these occur when starting or configuring a Squid
    component it must be resolved before you can run Squid.
  • ERROR messages indicate a serious problem which has broken an
    individual client transaction and may have some effect on other
    clients indirectly. But has not completely aborted all traffic
    service.
  • These can also occur when starting or configuring Squid
    components. In which case any service actions which that
    component would have supplied will not happen until it is
    resolved and Squid reconfigured.

    :information_source: Some log level 0 error messages inherited from older Squid
    versions exist without any prioritization tag.

  • WARNING messages indicate problems which might be causing
    problems to the client, but Squid is capable of working around
    automatically. These usually only display at log level 1 and higher.

    :information_source: Some log level 1 warning messages inherited from older
    Squid versions exist without any prioritization tag.

  • SECURITY ERROR messages indicate problems processing a client
    request with the security controls which Squid has been configured
    with. Some impossible condition is required to pass the security
    test.
    This is commonly seen when testing whether to accept a client
    request based on some reply detail which will only be
    available in the future.
  • SECURITY ALERT messages indicate security attack problems being
    detected. This is only for problems which are unambiguous. ‘Attacks’
    signatures which can appear in normal traffic are logged as regular
    WARNING.

    • A complete solution to these usually requires fixing the client,
      which may not be possible.
    • Administrative workarounds (extra firewall rules etc) can assist
      Squid in reducing the damage to network performance.
    • Attack notices may seem rather critical, but occur at level 1
      since in all cases Squid also has some workaround it can
      perform.
  • SECURITY NOTICE messages can appear during startup and
    reconfigure to indicate security related problems with the
    configuration file setting. These are accompanied by hints for
    better configuration where possible, and an indication of what Squid
    is going to do instead of the configured action.

Some of the more frequently questioned messages and what they mean are
outlined in the KnowledgeBase:

  1. Excess Data
  2. Failed to select source
  3. Host Header Forgery
  4. Queue congestion
  5. Too Many Queued Requests
  6. Unparseable Header

🔗 access.log

Most log file analysis program are based on the entries in access.log.

Squid allows the administrators to configure their
logfile format
and log output method
with great flexibility.

🔗 Squid result codes

The Squid result code is composed of several tags (separated by
underscore characters) which describe the response sent to the client.

  • One of these tags always exists to describe how it was delivered:

    TCP Requests on the HTTP port (usually 3128).
    UDP Requests on the ICP port (usually 3130) or HTCP port (usually 4128). If ICP logging was disabled using the log_icp_queries option, no ICP replies will be logged.
    NONE Squid delivered an unusual response or no response at all. Seen with cachemgr requests and errors, usually when the transaction fails before being classified into one of the above outcomes. Also seen with responses to CONNECT requests.

  • These tags are optional and describe why the particular handling was
    performed or where the request came from:

    CF At least one request in this transaction was collapsed. See collapsed_forwarding for more details about request collapsing. Support for this tag has been added to Squid v5 on 2018-06-18 (commit d2a6dc). It may not be available in earlier Squid versions.
    CLIENT The client request placed limits affecting the response. Usually seen with client issued a “no-cache”, or analogous cache control command along with the request. Thus, the cache has to validate the object.
    IMS The client sent a revalidation (conditional) request.
    ASYNC The request was generated internally by Squid. Usually this is background fetches for cache information exchanges, background revalidation from stale-while-revalidate cache controls, or ESI sub-objects being loaded.
    SWAPFAIL The object was believed to be in the cache, but could not be accessed. A new copy was requested from the server.
    REFRESH A revalidation (conditional) request was sent to the server.
    SHARED This tag is not supported yet. This request was combined with an existing transaction by collapsed forwarding. NOTE: the existing request is not marked as SHARED.
    REPLY The HTTP reply from server or peer. Usually seen on DENIED due to http_reply_access ACLs preventing delivery of servers response object to the client.

  • These tags are optional and describe what type of object was produced:

    NEGATIVE Only seen on HIT responses. Indicating the response was a cached error response. e.g. “404 not found”
    STALE The object was cached and served stale. This is usually caused by stale-while-revalidate or stale-if-error cache controls.
    OFFLINE The requested object was retrieved from the cache during offline_mode. The offline mode never validates any object.
    INVALID An invalid request was received. An error response was delivered indicating what the problem was.
    FAIL Only seen on REFRESH to indicate the revalidation request failed. The response object may be the server provided network error or the stale object which was being revalidated depending on stale-if-error cache control.
    MODIFIED Only seen on REFRESH responses to indicate revalidation produced a new modified object.
    UNMODIFIED Only seen on REFRESH responses to indicate revalidation produced a 304 (Not Modified) status. The client gets either a full 200 (OK), a 304 (Not Modified), or (in theory) another response, depending on the client request and other details.
    REDIRECT Squid generated an HTTP redirect response to this request.

  • These tags are optional and describe whether the response was loaded
    from cache, network, or otherwise:

    HIT The response object delivered was the local cache object.
    MEM Additional tag indicating the response object came from memory cache, avoiding disk accesses. Only seen on HIT responses.
    MISS The response object delivered was the network response object.
    DENIED The request was denied by access controls.
    NOFETCH A ICP specific type. Indicating service is alive, but not to be used for this request. Sent during “-Y” startup, or during frequent failures, a cache in hit only mode will return either UDP_HIT or UDP_MISS_NOFETCH. Neighbours will thus only fetch hits.
    TUNNEL A binary tunnel was established for this transaction.

  • These tags are optional and describe some error conditions which
    occured during response delivery (if any):

    ABORTED A client-to-Squid or Squid-to-server connection was closed unexpectedly, usually due to an I/O error or clean transport connection closure in the middle of some higher-level protocol message/negotiation. Before Squid v6, this tag was primarily seen when the client closed its connection to Squid before Squid could deliver the entire response. Since Squid v6, the tag also appears when Squid communication with an origin server or cache_peer is impossible (e.g., the server is refusing TCP connections) or aborted (e.g., an EOF in the middle of a chunked HTTP response body transfer).
    TIMEOUT The response was not completed due to a connection timeout.
    IGNORED While refreshing a previously cached response A, Squid got a response B that was older than A (as determined by the Date header field). Squid ignored response B (and attempted to use A instead). This “ignore older responses” logic complies with RFC 7234 Section 4 requirement: a cache MUST use the most recent response (as determined by the Date header field).

🔗 HTTP status codes

These are taken from RFC 1945
(HTTP/1.0), 2616 (HTTP/1.1) and
verified for Squid. Squid uses almost all codes except 416 (Request
Range Not Satisfiable). Extra codes used in the Squid logs (but not live
traffic) include 000 for a result code being unavailable, and 600 to
signal an invalid header, a proxy error. Also, some definitions were
added as for RFC 2518 and
4918 (WebDAV). Yes, there are
really two entries for status code 424:

Status Description RFC(s)
000 Used mostly with UDP traffic. N/A
  Informational  
100 Continue 2616
101 Switching Protocols 2616
102 Processing 2518
  Successful Transaction  
200 OK 1945, 2616
201 Created 1945, 2616
202 Accepted 1945, 2616
203 Non-Authoritative Information 2616
204 No Content 1945, 2616, 4918
205 Reset Content 2616
206 Partial Content 2616
207 Multi Status 2518, 4918
  Redirection  
300 Multiple Choices 1945, 2616, 4918
301 Moved Permanently 1945, 2616, 4918
302 Moved Temporarily 1945, 2616, 4918
303 See Other 2616, 4918
304 Not Modified 1945, 2616
305 Use Proxy 2616, 4918
307 Temporary Redirect 2616, 4918
  Client Error  
400 Bad Request 1945, 2616, 4918
401 Unauthorized 1945, 2616
402 Payment Required 2616
403 Forbidden 1945, 2616, 4918
404 Not Found 1945, 2616
405 Method Not Allowed 2616
406 Not Acceptable 2616
407 Proxy Authentication Required 2616
408 Request Timeout 2616
409 Conflict 2616, 4918
410 Gone 2616
411 Length Required 2616
412 Precondition Failed 2616, 4918
413 Request Entity Too Large 2616
414 Request URI Too Large 2616, 4918
415 Unsupported Media Type 2616
416 Request Range Not Satisfiable 2616
417 Expectation Failed 2616
422 Unprocessable Entity 2518, 4918
424 Locked (broken WebDAV implementations??)
424 Failed Dependency 2518, 4918
433 Unprocessable Entity  
  Server Errors  
500 Internal Server Error 1945, 2616
501 Not Implemented 1945, 2616
502 Bad Gateway 1945, 2616
503 Service Unavailable 1945, 2616
504 Gateway Timeout 2616
505 HTTP Version Not Supported 2616
507 Insufficient Storage 2518, 4918
  Broken Server Software  
600 Squid: header parsing error  
601 Squid: header size overflow detected while parsing  
601 roundcube: software configuration error  
603 roundcube: invalid authorization  

🔗 Request methods

Squid recognizes several request methods as defined in RFC
2616 and RFC
2518 “HTTP Extensions for
Distributed Authoring – WEBDAV” extensions.

 method    defined    cachabil.  meaning
 --------- ---------- ---------- -------------------------------------------
 GET       HTTP/0.9   possibly   object retrieval and simple searches.
 HEAD      HTTP/1.0   possibly   metadata retrieval.
 POST      HTTP/1.0   CC or Exp. submit data (to a program).
 PUT       HTTP/1.1   never      upload data (e.g. to a file).
 DELETE    HTTP/1.1   never      remove resource (e.g. file).
 TRACE     HTTP/1.1   never      appl. layer trace of request route.
 OPTIONS   HTTP/1.1   never      request available comm. options.
 CONNECT   HTTP/1.1r3 never      tunnel SSL connection.
 ICP_QUERY Squid      never      used for ICP based exchanges.
 PURGE     Squid      never      remove object from cache.
 PROPFIND  rfc2518    ?          retrieve properties of an object.
 PROPATCH  rfc2518    ?          change properties of an object.
 MKCOL     rfc2518    never      create a new collection.
 COPY      rfc2518    never      create a duplicate of src in dst.
 MOVE      rfc2518    never      atomically move src to dst.
 LOCK      rfc2518    never      lock an object against modifications.
 UNLOCK    rfc2518    never      unlock an object.

Note that since Squid 3.1, methods not listed here (such as PATCH) are
supported “out of the box.”

🔗 Hierarchy Codes

  • NONE For TCP HIT, TCP failures, cachemgr requests and all UDP
    requests, there is no hierarchy information.
  • DIRECT The object was fetched from the origin server.
  • SIBLING_HIT The object was fetched from a sibling cache which
    replied with UDP_HIT.
  • PARENT_HIT The object was requested from a parent cache which
    replied with UDP_HIT.
  • DEFAULT_PARENT No ICP queries were sent. This parent was chosen
    because it was marked “default” in the config file.
  • SINGLE_PARENT The object was requested from the only parent
    appropriate for the given URL.
  • FIRST_UP_PARENT The object was fetched from the first parent in
    the list of parents.
  • NO_PARENT_DIRECT The object was fetched from the origin server,
    because no parents existed for the given URL.
  • FIRST_PARENT_MISS The object was fetched from the parent with the
    fastest (possibly weighted) round trip time.
  • CLOSEST_PARENT_MISS This parent was chosen, because it included
    the the lowest RTT measurement to the origin server. See also the
    closest-only peer configuration option.
  • CLOSEST_PARENT The parent selection was based on our own RTT
    measurements.
  • CLOSEST_DIRECT Our own RTT measurements returned a shorter time
    than any parent.
  • NO_DIRECT_FAIL The object could not be requested because of a
    firewall configuration, see also never_direct and related material,
    and no parents were available.
  • SOURCE_FASTEST The origin site was chosen, because the source ping
    arrived fastest.
  • ROUNDROBIN_PARENT No ICP replies were received from any parent. The
    parent was chosen, because it was marked for round robin in the config
    file and had the lowest usage count.
  • CACHE_DIGEST_HIT The peer was chosen, because the cache digest
    predicted a hit. This option was later replaced in order to distinguish
    between parents and siblings.
  • CD_PARENT_HIT The parent was chosen, because the cache digest
    predicted a hit.
  • CD_SIBLING_HIT The sibling was chosen, because the cache digest
    predicted a hit.
  • NO_CACHE_DIGEST_DIRECT This output seems to be unused?
  • CARP The peer was selected by CARP.
  • PINNED The server connection was pinned by NTLM or Negotiate
    authentication requirements.
  • ORIGINAL_DST The server connection was limited to the client
    provided destination IP. This occurs on interception proxies when Host
    security is enabled, or
    client_dst_passthru
    transparency is enabled.
  • ANY_OLD_PARENT (former ANY_PARENT?) Squid used the first
    considered-alive parent it could reach. This happens when none of the
    specific parent cache selection algorithms (e.g., userhash or carp) were
    enabled, all enabled algorithms failed to find a suitable parent, or all
    suitable parents found by those algorithms failed when Squid tried to
    forward the request to them.
  • INVALID CODE part of src/peer_select.c:hier_strings[].

Almost any of these may be preceded by ‘TIMEOUT_’ if the two-second
(default) timeout occurs waiting for all ICP replies to arrive from
neighbors, see also the icp_query_timeout configuration option.

The following hierarchy codes were removed from Squid-2:

code                  meaning
--------------------  -------------------------------------------------
PARENT_UDP_HIT_OBJ    hit objects are not longer available.
SIBLING_UDP_HIT_OBJ   hit objects are not longer available.
SSL_PARENT_MISS       SSL can now be handled by squid.
FIREWALL_IP_DIRECT    No special logging for hosts inside the firewall.
LOCAL_IP_DIRECT       No special logging for local networks.

🔗 store.log

This file covers the objects currently kept on disk or removed ones. As
a kind of transaction log (or journal) it is usually used for debugging
purposes. A definitive statement, whether an object resides on your
disks is only possible after analyzing the complete log file. The
release (deletion) of an object may be logged at a later time than the
swap out (save to disk).

The store.log file may be of interest to log file analysis which looks
into the objects on your disks and the time they spend there, or how
many times a hot object was accessed. The latter may be covered by
another log file, too. With knowledge of the cache_dir configuration
option, this log file allows for a URL to filename mapping without
recursing your cache disks. However, the Squid developers recommend to
treat store.log primarily as a debug file, and so should you, unless
you know what you are doing.

The print format for a store log entry (one line) consists of thirteen
space-separated columns, compare with the storeLog() function in file
src/store_log.c:

9ld.%03d %-7s %02d %08X %s %4d %9ld %9ld %9ld %s %ld/%ld %s %s
  1. time The timestamp when the line was logged in UTC with a
    millisecond fraction.
  2. action The action the object was sumitted to, compare with
    src/store_log.c:

    • CREATE Seems to be unused.
    • RELEASE The object was removed from the cache (see also
      file number below).
    • SWAPOUT The object was saved to disk.
    • SWAPIN The object existed on disk and was read into memory
  3. dir number The cache_dir number this object was stored into,
    starting at 0 for your first cache_dir line.
  4. file number The file number for the object storage file. Please
    note that the path to this file is calculated according to your
    cache_dir configuration. A file number of FFFFFFFF indicates
    “memory only” objects. Any action code for such a file number refers
    to an object which existed only in memory, not on disk. For
    instance, if a RELEASE code was logged with file number
    FFFFFFFF, the object existed only in memory, and was released from
    memory.
  5. hash The hash value used to index the object in the cache. Squid
    currently uses MD5 for the hash value.
  6. status The HTTP reply status code.
  7. datehdr The value of the HTTP Date reply header.
  8. lastmod The value of the HTTP Last-Modified reply header.
  9. expires The value of the HTTP “Expires: “ reply header.
  10. type The HTTP Content-Type major value, or “unknown” if it
    cannot be determined.
  11. sizes This column consists of two slash separated fields:

    • The advertised content length from the HTTP Content-Length
      reply header.
    • The size actually read.
    • If the advertised (or expected) length is missing, it will
      be set to zero. If the advertised length is not zero, but
      not equal to the real length, the object will be released
      from the cache.
  12. method The request method for the object, e.g. GET.
  13. key The key to the object, usually the URL.

    • The datehdr, lastmod, and expires values are all
      expressed in UTC seconds. The actual values are parsed from the
      HTTP reply headers. An unparsable header is represented by a
      value of -1, and a missing header is represented by a value of
      -2.

🔗 swap.state

This file has a rather unfortunate history which has led to it often
being called the swap log. It is in fact a journal of the cache
index with a record of every cache object written to disk. It is read
when Squid starts up to “reload” the cache quickly.

If you remove this file when squid is NOT running, you will
effectively wipe out your cache index of contents. Squid can rebuild it
from the original files, but that procedure can take a long time as
every file in the cache must be fully scanned for meta data.

If you remove this file while squid IS running, you can easily
recreate it. The safest way is to simply shutdown the running process:

This will disrupt service, but at least you will have your swap log
back. Alternatively, you can tell squid to rotate its log files. This
also causes a clean swap log to be written.

By default the swap.state file is stored in the top-level of each
cache_dir. You can move the logs to a different location with the
cache_swap_state option.

The file is a binary format that includes MD5 checksums, and
StoreEntry fields. Please see the Programmers’ Guide for information
on the contents and format of that file.

🔗 squid.out

If you run your Squid from the RunCache script, a file squid.out
contains the Squid startup times, and also all fatal errors, e.g. as
produced by an assert() failure. If you are not using RunCache, you
will not see such a file.

:warning: RunCache
has been obsoleted since Squid-2.6.
Modern Squid run as daemons usually log this output to the system
syslog facility or if run manually to stdout for the account which
operates the master daemon process.

🔗 useragent.log

:warning:
Starting from
Squid-3.2
this log has become one of the default access.log
formats and is always available for use. It is no longer a special
separate log file.

🔗 Which log files can I delete safely?

You should never delete access.log, store.log, or cache.log while
Squid is running. With Unix, you can delete a file when a process has
the file opened. However, the filesystem space is not reclaimed until
the process closes the file.

If you accidentally delete swap.state while Squid is running, you can
recover it by following the instructions in the previous questions. If
you delete the others while Squid is running, you can not recover them.

The correct way to maintain your log files is with Squid’s “rotate”
feature. You should rotate your log files at least once per day. The
current log files are closed and then renamed with numeric extensions
(.0, .1, etc). If you want to, you can write your own scripts to archive
or remove the old log files. If not, Squid will only keep up to
logfile_rotate
versions of each log file. The logfile rotation procedure also writes a
clean swap.state file, but it does not leave numbered versions of the
old files.

If you set
logfile_rotate
to 0, Squid simply closes and then re-opens the logs. This allows
third-party logfile management systems, such as newsyslog, to maintain
the log files.

To rotate Squid’s logs, simple use this command:

For example, use this cron entry to rotate the logs at midnight:

0 0 * * * /usr/local/squid/bin/squid -k rotate

🔗 How can I disable Squid’s log files?

To disable access.log:

To disable store.log:

To disable cache.log:

:warning:
It is a bad idea to disable the cache.log because this file contains
many important status and debugging messages.
However, if you really want to, you can

:warning:
If /dev/null is specified to any of the above log files,
logfile_rotate MUST
also be set to 0 or else risk Squid rotating away /dev/null making
t a plain log file

:information_source:
Instead of disabling the log files, it is advisable to use a smaller value
for logfile_rotate
and properly rotating Squid’s log files in your cron.
That way, your log files are more controllable and
self-maintained by your system

🔗 What is the maximum size of access.log?

Squid does not impose a size limit on its log files. Some operating
systems have a maximum file size limit, however. If a Squid log file
exceeds the operating system’s size limit, Squid receives a write error
and shuts down. You should regularly rotate Squid’s log files so that
they do not become very large.

:warning:
Logging is very important to Squid.
In fact, it is so important that it will shut itself down if it
can’t write to its logfiles. This includes cases such as a full log disk,
or logfiles getting too big.

🔗 My log files get very big!

You need to rotate your log files with a cron job. For example:

0 0 * * * /usr/local/squid/bin/squid -k rotate

When logging debug information into cache.log it can easily become
extremely large and when a long access.log traffic history is required
(ie by law in some countries) storing large cache.log for that time is
not reasonable. From Squid-3.2
cache.log can be rotated with an individual cap set by
debug_options
rotate=N} option to store fewer of these large files in the .0 to .N
series of backups. The default is to store the same number as with
access.log and set in the
logfile_rotate
directive.

🔗 I want to use another tool to maintain the log files.

If you set
logfile_rotate
to 0, Squid simply closes and then re-opens the logs. This allows
third-party logfile management systems, such as
newsyslog or
logrotate, to maintain the log files.

🔗 Managing log files

The preferred log file for analysis is the access.log file in native
format. For long term evaluations, the log file should be obtained at
regular intervals. Squid offers an easy to use API for rotating log
files, in order that they may be moved (or removed) without disturbing
the cache operations in progress. The procedures were described above.

Depending on the disk space allocated for log file storage, it is
recommended to set up a cron job which rotates the log files every 24,
12, or 8 hour. You will need to set your
logfile_rotate
to a sufficiently large number. During a time of some idleness, you can
safely transfer the log files to your analysis host in one burst.

Before transport, the log files can be compressed during off-peak time.
On the analysis host, the log file are concatenated into one file, so
one file for 24 hours is the yield. Also note that with
log_icp_queries
enabled, you might have around 1 GB of uncompressed log information per
day and busy cache. Look into you cache manager info page to make an
educated guess on the size of your log files.

Some basic advice to obey when handling and processing log files:

  • Respect the privacy of your clients when publishing results.
  • Keep logs unavailable unless anonymized. Most countries have laws on
    privacy protection, and some even on how long you are legally
    allowed to keep certain kinds of information.
  • Rotate and process log files at least once a day. Even if you don’t
    process the log files, they will grow quite large, see My log files
    get very big     above here. If you rely on processing the log files,
    reserve a large enough partition solely for log files.
  • Keep the size in mind when processing. It might take longer to
    process log files than to generate them!
  • Limit yourself to the numbers you are interested in. There is data
    beyond your dreams available in your log file, some quite obvious,
    others by combination of different views. Here are some examples for
    figures to watch:

    • The hosts using your cache.
    • The elapsed time for HTTP requests — this is the latency the
      user sees. Usually, you will want to make a distinction for HITs
      and MISSes and overall times. Also, medians are preferred over
      averages.
    • The requests handled per interval (e.g. second, minute or hour).

🔗 Why do I get ERR_NO_CLIENTS_BIG_OBJ messages so often?

This message means that the requested object was in “Delete Behind” mode
and the user aborted the transfer. An object will go into “Delete
Behind” mode if

  • It is larger than maximum_object_size
  • It is being fetched from a neighbor which has the proxy-only
    option set

🔗 What does ERR_LIFETIME_EXP mean?

This means that a timeout occurred while the object was being
transferred. Most likely the retrieval of this object was very slow (or
it stalled before finishing) and the user aborted the request. However,
depending on your settings for quick_abort, Squid may have continued
to try retrieving the object. Squid imposes a maximum amount of time on
all open sockets, so after some amount of time the stalled request was
aborted and logged win an ERR_LIFETIME_EXP message.

🔗 Retrieving “lost” files from the cache

“I’ve been asked to retrieve an object which was accidentally destroyed
at the source for recovery. So, how do I figure out where the things are
so I can copy them out and strip off the headers?””

The following method applies only to the Squid-1.1 versions:

Use grep to find the named object (URL) in the cache.log file. The
first field in this file is an integer file number.

Then, find the file fileno-to-pathname.pl from the “scripts” directory
of the Squid source distribution. The usage is

perl fileno-to-pathname.pl [-c squid.conf]

file numbers are read on stdin, and pathnames are printed on stdout.

🔗 Can I use store.log to figure out if a response was cachable?

Sort of. You can use store.log to find out if a particular response
was cached.

Cached responses are logged with the SWAPOUT tag. Uncached responses are
logged with the RELEASE tag.

However, your analysis must also consider that when a cached response is
removed from the cache (for example due to cache replacement) it is also
logged in store.log with the RELEASE tag. To differentiate these two,
you can look at the filenumber (3rd) field. When an uncachable response
is released, the filenumber is FFFFFFFF (-1). Any other filenumber
indicates a cached response was released.

🔗 Can I pump the squid access.log directly into a pipe?

Several people have asked for this, usually to feed the log into some
kind of external database, or to analyze them in real-time.

The answer is No. Well, yes, sorta. Using a pipe directly opens up a
whole load of possible problems.

:warning:
Logging is very important to Squid.
In fact, it is so important that it will shut itself down if it
can’t write to its logfiles.

There are several alternatives which are much safer to setup and use.
The basic capabilities present are :

See the Log Modules feature
for technical details on setting up a daemon or other output modules.

To the FAQ Index

Navigation: Site Search,
Site Pages,
Categories, 🔼 go up

Источник

Squid access log is very informative if you know how to dig something out of it. Following is an a line from access.log file.

1201172176.719 1190 127.0.0.1 TCP_MISS/200 529 GET http://www.blogger.com/status.g? — DIRECT/72.14.221.191 application/xmlThis line can be written as:

Timestamp, Total time, Source, Action/Code, Size, Method, URL, Ident,

Hierarchy/From, Content type

1: Timestamp Time when the request was completed.
2: Total time Total time taken to complete the request
3: Source IP address of the client
4: Action/Code Action taken for the request
5: Size Total size of the request in bytes
6: Method Whether the request was GET or POST
7: URL The actual request
8: Ident Usually —
9: Hierarchy/From How the object is fetched and from where
10: Content type Type of object


Action

«TCP_» refers to requests on the HTTP port (3128)
TCP_HIT A valid copy of the requested object was in the cache.
TCP_MISS The requested object was not in the cache.
TCP_REFRESH_HIT An expired copy of the requested object was in the cache. Squid made an If-Modified-Since request and the response was «Not Modified.»
TCP_REFRESH_FAIL_HIT An expired copy of the requested object was in the cache. Squid attempted to make an If-Modified-Since request, but it failed. The old (stale) object was delivered to the client.
TCP_REFRESH_MISS An expired copy of the requested object was in the cache. Squid made an If-Modified-Since request and received a new, different object.
TCP_CLIENT_REFRESH The client issued a request with the «no-cache» pragma. («reload» — handled as MISS)
TCP_IMS_HIT An If-Modified-Since GET request was received from the client. A valid copy of the object was in the cache (fresh).
TCP_IMS_MISS An If-Modified-Since GET request was received from the client. The requested object was not in the cache (stale).
TCP_SWAPFAIL The object was believed to be in the cache, but could not be accessed.
TCP_DENIED Access was denied for this request.
«UDP_» refers to requests on the ICP port (3130)
UDP_HIT A valid copy of the requested object was in the cache
UDP_HIT_OBJ Same as UDP_HIT, but the object data was small enough to be sent in the UDP reply packet. Saves the following TCP request.
UDP_MISS The requested object was not in the cache
UDP_DENIED Access was denied for this request
UDP_INVALID An invalid request was received.
UDP_RELOADING The neighbor cache is reloading its disk store metadata and does not want any TCP requests for MISSES until it is finished.
Errors
ERR_READ_TIMEOUT The remote site or network is unreachable — may be down.
ERR_LIFETIME_EXP The remote site or network may be too slow or down.
ERR_NO_CLIENTS_BIG_OBJ All Clients went away before tranmission completed and the object is too big to cache.
ERR_READ_ERROR The remote site or network may be down.
ERR_CLIENT_ABORT Client dropped connection before transmission completed. Squid fetches the Object according to its settings for `quick_abort’.
ERR_CONNECT_FAIL The remote site or server may be down.
ERR_INVALID_REQ Invalid HTTP request
ERR_UNSUP_REQ Unsupported request
ERR_INVALID_URL Invalid URL syntax
ERR_NO_FDS Out of file descriptors
ERR_DNS_FAIL DNS name lookup failure
ERR_NOT_IMPLEMENTED Protocol Not Supported
ERR_CANNOT_FETCH The requested URL can not currently be retrieved.
ERR_NO_RELAY There is no WAIS relay host defined for this cache.
ERR_DISK_IO The system disk is out of space or failing.
ERR_ZERO_SIZE_OBJECT The remote server closed the connection before sending any data.
ERR_FTP_DISABLED This cache is configured to NOT retrieve FTP objects.
ERR_PROXY_DENIED Access Denied. The user must authenticate himself before accessing this cache.


CODE

Code Reason phrase RFC 2616 section
0 No Response Received (Squid-specific) N/A
1xx Informational 10.1
100 Continue 10.1.1
101 Switching Protocols 10.1.2
2xx Successful 10.2
200 OK 10.2.1
201 Created 10.2.2
202 Accepted 10.2.3
203 Non-Authoritative Information 10.2.4
204 No Content 10.2.5
205 Reset Content 10.2.6
206 Partial Content 10.2.7
3xx Redirection 10.3
300 Multiple Choices 10.3.1
301 Moved Permanently 10.3.2
302 Found 10.3.3
303 See Other 10.3.4
304 Not Modified 10.3.5
305 Use Proxy 10.3.6
306 (Unused) 10.3.7
307 Temporary Redirect 10.3.8
4xx Client Error 10.4
400 Bad Request 10.4.1
401 Unauthorized 10.4.2
402 Payment Required 10.4.3
403 Forbidden 10.4.4
404 Not Found 10.4.5
405 Method Not Allowed 10.4.6
406 Not Acceptable 10.4.7
407 Proxy Authentication Required 10.4.8
408 Request Timeout 10.4.9
409 Conflict 10.4.10
410 Gone 10.4.11
411 Length Required 10.4.12
412 Precondition Failed 10.4.13
413 Request Entity Too Large 10.4.14
414 Request-URI Too Long 10.4.15
415 Unsupported Media Type 10.4.16
416 Requested Range Not Satisfiable 10.4.17
417 Expectation Failed 10.4.18
5xx Server Error 10.5
500 Internal Server Error 10.5.1
501 Not Implemented 10.5.2
502 Bad Gateway 10.5.3
503 Service Unavailable 10.5.4
504 Gateway Timeout 10.5.5
505 HTTP Version Not Supported 10.5.6
6xx Proxy Error N/A
600 Unparseable Response Headers (Squid-specific) N/A

Methods

GET Request URL
HEAD Request only HTTP headers of the supplied URL and no document body
POST Transfer data to the supplied URL
PUT Store data under the supplied URL
CONNECT Forward data to SSL-Server:Port
ICP_QUERY Request from a Parent/Neighbor for the supplied URL
NONE Request of an unsupported method

Hierarchy

NONE The object requested by a sibling, was not in my cache.
DIRECT The object has been requested from the origin server.
SIBLING_HIT The object was requested from a neighbor cache which replied with a UDP_HIT (formerly logged as NEIGHBOR_HIT).
PARENT_HIT The object was requested from a parent cache which replied with a UDP_HIT.
DEFAULT_PARENT The object was requested from a default parent cache appropriate for this URL.
SINGLE_PARENT The object was requested from the only parent cache appropriate for this URL.
FIRST_UP_PARENT The object has been requested from the first available parent in your list.
NO_PARENT_DIRECT The object was requested from the origin server because no parent caches exist for the URL.
FIRST_PARENT_MISS The object has been requested from the parent cache with the fastest weighted round trip time.
ROUNDROBIN_PARENT No ICP queries were received from any parent caches. This parent was chosen because it was marked as ‘default’ in the config file and it had the lowest round-robin use count.
CLOSEST_PARENT_MISS This parent was selected because it included the lowest RTT measurement to the origin server. This only appears with ‘query_icmp on’ set in the config file.
CLOSEST_DIRECT The object was fetched directly from the origin server because this cache measured a lower RTT than any of the parent caches.
LOCAL_IP_DIRECT The object has been requested from the origin server because the origin host IP address matched your ‘local_ip’ list.
FIREWALL_IP_DIRECT The object has been requested from the origin server because the origin host IP address is inside your firewall.
NO_DIRECT_FAIL The object could not be requested because of firewall restrictions and no parent caches were available.
SOURCE_FASTEST The object was requested from the origin server because the ‘source_ping’ reply arrived first.
SIBLING_UDP_HIT_OBJ The object was received in a UDP_HIT_OBJ reply from a neighbor cache (formerly logged as UDP_HIT_OBJ).
PARENT_UDP_HIT_OBJ The object was received in a UDP_HIT_OBJ reply from a parent cache (formerly logged as UDP_HIT_OBJ).
PASSTHROUGH_PARENT The neighbor or proxy defined in the config option ‘passthrough_proxy’ was used.
SSL_PARENT_MISS The neighbor or proxy defined in the config option ‘ssl_proxy’ was used.
Источник

Чтобы каждый раз не искать, решил закинуть сюда

Коды TCP_ соотвествуют запросам на HTTP-порту (обычно 3128). Коды
UDP_ соотвествуют запросам на ICP-порту (обычно 3130). Если журналирование
ICP было выключено при помощи опции log_icp_queries, то ICP-ответы попадать в журнал не будут.

Нижеследующие результирующие коды были взяты из Squid-2, в соответствии со структурой
log_tags в файле src/access_log.c:

TCP_HIT
Верная копия запрошенного объекта была в кеше.

TCP_MISS
Запрошенного объекта не было в кеше.

TCP_REFRESH_HIT
Запрошенный объект был закеширован, но УСТАРЕЛ.
IMS-запрос для этого объекта вернул “304 not modified”.

TCP_REF_FAIL_HIT
Запрошенный объект был закеширован, но УСТАРЕЛ.
IMS-запрос завершен неудачно и устаревший объект был доставлен.

TCP_REFRESH_MISS
Запрошенный объект был закеширован, но УСТАРЕЛ.
IMS-запрос вернул новое содержимое.

TCP_CLIENT_REFRESH_MISS
Клиент послал прагму “no-cache” или другу аналогичную команду контроля кеширования в запросе. Поэтому
кеш должен повторно получить объект.

TCP_IMS_HIT
Клиент использовал IMS-запрос для объекта, который был найден в кеше свежим.

TCP_SWAPFAIL_MISS
Объект скорее всего был в кеше, но доступа к нему нет.

TCP_NEGATIVE_HIT
Запрос для негативно кешированных объектов типа
“404 not found”, о которых кеш знает, что они недоступны. См. пояснения по negative_ttl
в вашем файле squid.conf.

TCP_MEM_HIT
Верная копия запрошенного объекта была в кеше
и в памяти, доступа к диску не производилось.

TCP_DENIED
Доступ запрещен для этого запроса.

TCP_OFFLINE_HIT
Запрошенный объект был извлечен из кеша в режиме
offline. В режиме offline никогда не проверяются, см. offline_mode в файле squid.conf.

UDP_HIT
Верная копия запрошенного объекта была в кеше.

UDP_MISS
Запрошенный объект отсутствует в этом кеше.

UDP_DENIED
Доступ запрещен для этого запроса.

UDP_INVALID
Был получен неверный запрос.

UDP_MISS_NOFETCH
Из-за опции запуска “-Y” или частых отказов, кеш при хите будет возвращать либо UDP_HIT или этот код.
Соседи таким образом получат только хиты.

NONE
Указывается с ошибками и запросами cachemgr.

Следующие коды больше недоступны в Squid-2:

ERR_*
Ошибки теперь указываются в статусе кода.

TCP_CLIENT_REFRESH
Смотреть в TCP_CLIENT_REFRESH_MISS

TCP_SWAPFAIL
Смотреть в TCP_SWAPFAIL_MISS.

TCP_IMS_MISS
Удалено, вместо этого используется TCP_IMS_HIT

UDP_HIT_OBJ
Совпавший объект больше недоступен.

UDP_RELOADING
Смотреть в UDP_MISS_NOFETCH

Спасибо ресурсу http://saygak.com/post/64

Источник

Формат лога

Разберемся с форматом ‘по умолчанию’ 

#logformat squid %ts.%03tu %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt
access_log /var/log/squid/access.log squid

Знак % означат что далее следует литерал а не терминальный символ

  • %ts.%03tu : точка — это терминальный символ разделяющий значения, а число 03 аргумент (т.е. точность до 3х знаков) для tu

ts : Seconds since epoch — Количество секунд от 00:00 1970.01.01

tu : subsecond time (milliseconds) — доли секунду

  • %6tr : Time Response — время отклика, миллисекунд

  • %>a : адрес источника запроса

  • %Ss/%03Hs :

Ss : Squid request status

Статус запроса в squid

Hs : HTTP status code — Статус коды HTTP протокола

  • %<st : передано байт (В ответ, включая HTTP заголовок)

  • %rm : Request method — метод запроса (POST, GET …)

  • %ru : Request URL

  • %un : User name

  • %Sh/%<A :

Sh : Squid hierarchy status

Коды иерархий в Squid

<A : IP адрес запрашиваемого сервера

  • %mt : MIME type

Создание собственного формата

Первое что хотелось бы сделать — это привычный формат времени

%tl — Local time, по умолчанию получим запись вот такого вида %d/%b/%Y:%H:%M:%S %z 

07/Aug/2009:16:06:04 +0400

%tg — GMT time, покажет мировое время 

07/Aug/2009:12:06:04 +0000

Модифицируем в %{%Y.%m.%d/%H:%M:%S}tl и получим 

2009.08.07/16:26:55

Разделение логов

При необходимости можно легко разделять логи для разных acl 

access_log <file> [<logformat name> [acl acl ...]]

Пример разделения логов с локального хоста и из локальной сети 

access_log /var/log/squid/access.log mylogformat localhost
access_log /var/log/squid/access_parsing.log mylogformat localnet

Чтобы логировать полый адрес URL 

strip_query_terms off
Источник

The logs are a valuable source of information about Squid workloads and
performance. The logs record not only access information, but also
system configuration errors and resource consumption (e.g. memory, disk
space). There are several log file maintained by Squid. Some have to be
explicitly activated during compile time, others can safely be
deactivated during run-time.

There are a few basic points common to all log files. The time stamps
logged into the log files are usually UTC seconds unless stated
otherwise. The initial time stamp usually contains a millisecond
extension.

🔗 cache.log

The cache.log file contains the debug and error messages that Squid
generates. If you start your Squid using the -s command line option, a
copy of certain messages will go into your syslog facilities. It is a
matter of personal preferences to use a separate file for the squid log
data.

From the area of automatic log file analysis, the cache.log file does
not have much to offer. You will usually look into this file for
automated error reports, when programming Squid, testing new features,
or searching for reasons of a perceived misbehavior, etc.

🔗 Squid Error Messages

Error messages come in several forms. Debug traces are not logged at
level 0 or level 1. These levels are reserved for important and critical
administrative messages.

  • FATAL messages indicate a problem which has killed the Squid
    process. Affecting all current client traffic being supplied by that
    Squid instance.
    If these occur when starting or configuring a Squid
    component it must be resolved before you can run Squid.
  • ERROR messages indicate a serious problem which has broken an
    individual client transaction and may have some effect on other
    clients indirectly. But has not completely aborted all traffic
    service.
  • These can also occur when starting or configuring Squid
    components. In which case any service actions which that
    component would have supplied will not happen until it is
    resolved and Squid reconfigured.

    :information_source: Some log level 0 error messages inherited from older Squid
    versions exist without any prioritization tag.

  • WARNING messages indicate problems which might be causing
    problems to the client, but Squid is capable of working around
    automatically. These usually only display at log level 1 and higher.

    :information_source: Some log level 1 warning messages inherited from older
    Squid versions exist without any prioritization tag.

  • SECURITY ERROR messages indicate problems processing a client
    request with the security controls which Squid has been configured
    with. Some impossible condition is required to pass the security
    test.
    This is commonly seen when testing whether to accept a client
    request based on some reply detail which will only be
    available in the future.
  • SECURITY ALERT messages indicate security attack problems being
    detected. This is only for problems which are unambiguous. ‘Attacks’
    signatures which can appear in normal traffic are logged as regular
    WARNING.

    • A complete solution to these usually requires fixing the client,
      which may not be possible.
    • Administrative workarounds (extra firewall rules etc) can assist
      Squid in reducing the damage to network performance.
    • Attack notices may seem rather critical, but occur at level 1
      since in all cases Squid also has some workaround it can
      perform.
  • SECURITY NOTICE messages can appear during startup and
    reconfigure to indicate security related problems with the
    configuration file setting. These are accompanied by hints for
    better configuration where possible, and an indication of what Squid
    is going to do instead of the configured action.

Some of the more frequently questioned messages and what they mean are
outlined in the KnowledgeBase:

  1. Excess Data
  2. Failed to select source
  3. Host Header Forgery
  4. Queue congestion
  5. Too Many Queued Requests
  6. Unparseable Header

🔗 access.log

Most log file analysis program are based on the entries in access.log.

Squid allows the administrators to configure their
logfile format
and log output method
with great flexibility.

🔗 Squid result codes

The Squid result code is composed of several tags (separated by
underscore characters) which describe the response sent to the client.

  • One of these tags always exists to describe how it was delivered:

    TCP Requests on the HTTP port (usually 3128).
    UDP Requests on the ICP port (usually 3130) or HTCP port (usually 4128). If ICP logging was disabled using the log_icp_queries option, no ICP replies will be logged.
    NONE Squid delivered an unusual response or no response at all. Seen with cachemgr requests and errors, usually when the transaction fails before being classified into one of the above outcomes. Also seen with responses to CONNECT requests.

  • These tags are optional and describe why the particular handling was
    performed or where the request came from:

    CF At least one request in this transaction was collapsed. See collapsed_forwarding for more details about request collapsing. Support for this tag has been added to Squid v5 on 2018-06-18 (commit d2a6dc). It may not be available in earlier Squid versions.
    CLIENT The client request placed limits affecting the response. Usually seen with client issued a “no-cache”, or analogous cache control command along with the request. Thus, the cache has to validate the object.
    IMS The client sent a revalidation (conditional) request.
    ASYNC The request was generated internally by Squid. Usually this is background fetches for cache information exchanges, background revalidation from stale-while-revalidate cache controls, or ESI sub-objects being loaded.
    SWAPFAIL The object was believed to be in the cache, but could not be accessed. A new copy was requested from the server.
    REFRESH A revalidation (conditional) request was sent to the server.
    SHARED This tag is not supported yet. This request was combined with an existing transaction by collapsed forwarding. NOTE: the existing request is not marked as SHARED.
    REPLY The HTTP reply from server or peer. Usually seen on DENIED due to http_reply_access ACLs preventing delivery of servers response object to the client.

  • These tags are optional and describe what type of object was produced:

    NEGATIVE Only seen on HIT responses. Indicating the response was a cached error response. e.g. “404 not found”
    STALE The object was cached and served stale. This is usually caused by stale-while-revalidate or stale-if-error cache controls.
    OFFLINE The requested object was retrieved from the cache during offline_mode. The offline mode never validates any object.
    INVALID An invalid request was received. An error response was delivered indicating what the problem was.
    FAIL Only seen on REFRESH to indicate the revalidation request failed. The response object may be the server provided network error or the stale object which was being revalidated depending on stale-if-error cache control.
    MODIFIED Only seen on REFRESH responses to indicate revalidation produced a new modified object.
    UNMODIFIED Only seen on REFRESH responses to indicate revalidation produced a 304 (Not Modified) status. The client gets either a full 200 (OK), a 304 (Not Modified), or (in theory) another response, depending on the client request and other details.
    REDIRECT Squid generated an HTTP redirect response to this request.

  • These tags are optional and describe whether the response was loaded
    from cache, network, or otherwise:

    HIT The response object delivered was the local cache object.
    MEM Additional tag indicating the response object came from memory cache, avoiding disk accesses. Only seen on HIT responses.
    MISS The response object delivered was the network response object.
    DENIED The request was denied by access controls.
    NOFETCH A ICP specific type. Indicating service is alive, but not to be used for this request. Sent during “-Y” startup, or during frequent failures, a cache in hit only mode will return either UDP_HIT or UDP_MISS_NOFETCH. Neighbours will thus only fetch hits.
    TUNNEL A binary tunnel was established for this transaction.

  • These tags are optional and describe some error conditions which
    occured during response delivery (if any):

    ABORTED The response was not completed due to the connection being aborted (usually by the client).
    TIMEOUT The response was not completed due to a connection timeout.
    IGNORED While refreshing a previously cached response A, Squid got a response B that was older than A (as determined by the Date header field). Squid ignored response B (and attempted to use A instead). This “ignore older responses” logic complies with RFC 7234 Section 4 requirement: a cache MUST use the most recent response (as determined by the Date header field).

🔗 HTTP status codes

These are taken from RFC 1945
(HTTP/1.0), 2616 (HTTP/1.1) and
verified for Squid. Squid uses almost all codes except 416 (Request
Range Not Satisfiable). Extra codes used in the Squid logs (but not live
traffic) include 000 for a result code being unavailable, and 600 to
signal an invalid header, a proxy error. Also, some definitions were
added as for RFC 2518 and
4918 (WebDAV). Yes, there are
really two entries for status code 424:

Status Description RFC(s)
000 Used mostly with UDP traffic. N/A
  Informational  
100 Continue 2616
101 Switching Protocols 2616
102 Processing 2518
  Successful Transaction  
200 OK 1945, 2616
201 Created 1945, 2616
202 Accepted 1945, 2616
203 Non-Authoritative Information 2616
204 No Content 1945, 2616, 4918
205 Reset Content 2616
206 Partial Content 2616
207 Multi Status 2518, 4918
  Redirection  
300 Multiple Choices 1945, 2616, 4918
301 Moved Permanently 1945, 2616, 4918
302 Moved Temporarily 1945, 2616, 4918
303 See Other 2616, 4918
304 Not Modified 1945, 2616
305 Use Proxy 2616, 4918
307 Temporary Redirect 2616, 4918
  Client Error  
400 Bad Request 1945, 2616, 4918
401 Unauthorized 1945, 2616
402 Payment Required 2616
403 Forbidden 1945, 2616, 4918
404 Not Found 1945, 2616
405 Method Not Allowed 2616
406 Not Acceptable 2616
407 Proxy Authentication Required 2616
408 Request Timeout 2616
409 Conflict 2616, 4918
410 Gone 2616
411 Length Required 2616
412 Precondition Failed 2616, 4918
413 Request Entity Too Large 2616
414 Request URI Too Large 2616, 4918
415 Unsupported Media Type 2616
416 Request Range Not Satisfiable 2616
417 Expectation Failed 2616
422 Unprocessable Entity 2518, 4918
424 Locked (broken WebDAV implementations??)
424 Failed Dependency 2518, 4918
433 Unprocessable Entity  
  Server Errors  
500 Internal Server Error 1945, 2616
501 Not Implemented 1945, 2616
502 Bad Gateway 1945, 2616
503 Service Unavailable 1945, 2616
504 Gateway Timeout 2616
505 HTTP Version Not Supported 2616
507 Insufficient Storage 2518, 4918
  Broken Server Software  
600 Squid: header parsing error  
601 Squid: header size overflow detected while parsing  
601 roundcube: software configuration error  
603 roundcube: invalid authorization  

🔗 Request methods

Squid recognizes several request methods as defined in RFC
2616 and RFC
2518 “HTTP Extensions for
Distributed Authoring – WEBDAV” extensions.

 method    defined    cachabil.  meaning
 --------- ---------- ---------- -------------------------------------------
 GET       HTTP/0.9   possibly   object retrieval and simple searches.
 HEAD      HTTP/1.0   possibly   metadata retrieval.
 POST      HTTP/1.0   CC or Exp. submit data (to a program).
 PUT       HTTP/1.1   never      upload data (e.g. to a file).
 DELETE    HTTP/1.1   never      remove resource (e.g. file).
 TRACE     HTTP/1.1   never      appl. layer trace of request route.
 OPTIONS   HTTP/1.1   never      request available comm. options.
 CONNECT   HTTP/1.1r3 never      tunnel SSL connection.
 ICP_QUERY Squid      never      used for ICP based exchanges.
 PURGE     Squid      never      remove object from cache.
 PROPFIND  rfc2518    ?          retrieve properties of an object.
 PROPATCH  rfc2518    ?          change properties of an object.
 MKCOL     rfc2518    never      create a new collection.
 COPY      rfc2518    never      create a duplicate of src in dst.
 MOVE      rfc2518    never      atomically move src to dst.
 LOCK      rfc2518    never      lock an object against modifications.
 UNLOCK    rfc2518    never      unlock an object.

Note that since Squid 3.1, methods not listed here (such as PATCH) are
supported “out of the box.”

🔗 Hierarchy Codes

  • NONE For TCP HIT, TCP failures, cachemgr requests and all UDP
    requests, there is no hierarchy information.
  • DIRECT The object was fetched from the origin server.
  • SIBLING_HIT The object was fetched from a sibling cache which
    replied with UDP_HIT.
  • PARENT_HIT The object was requested from a parent cache which
    replied with UDP_HIT.
  • DEFAULT_PARENT No ICP queries were sent. This parent was chosen
    because it was marked “default” in the config file.
  • SINGLE_PARENT The object was requested from the only parent
    appropriate for the given URL.
  • FIRST_UP_PARENT The object was fetched from the first parent in
    the list of parents.
  • NO_PARENT_DIRECT The object was fetched from the origin server,
    because no parents existed for the given URL.
  • FIRST_PARENT_MISS The object was fetched from the parent with the
    fastest (possibly weighted) round trip time.
  • CLOSEST_PARENT_MISS This parent was chosen, because it included
    the the lowest RTT measurement to the origin server. See also the
    closest-only peer configuration option.
  • CLOSEST_PARENT The parent selection was based on our own RTT
    measurements.
  • CLOSEST_DIRECT Our own RTT measurements returned a shorter time
    than any parent.
  • NO_DIRECT_FAIL The object could not be requested because of a
    firewall configuration, see also never_direct and related material,
    and no parents were available.
  • SOURCE_FASTEST The origin site was chosen, because the source ping
    arrived fastest.
  • ROUNDROBIN_PARENT No ICP replies were received from any parent. The
    parent was chosen, because it was marked for round robin in the config
    file and had the lowest usage count.
  • CACHE_DIGEST_HIT The peer was chosen, because the cache digest
    predicted a hit. This option was later replaced in order to distinguish
    between parents and siblings.
  • CD_PARENT_HIT The parent was chosen, because the cache digest
    predicted a hit.
  • CD_SIBLING_HIT The sibling was chosen, because the cache digest
    predicted a hit.
  • NO_CACHE_DIGEST_DIRECT This output seems to be unused?
  • CARP The peer was selected by CARP.
  • PINNED The server connection was pinned by NTLM or Negotiate
    authentication requirements.
  • ORIGINAL_DST The server connection was limited to the client
    provided destination IP. This occurs on interception proxies when Host
    security is enabled, or
    client_dst_passthru
    transparency is enabled.
  • ANY_OLD_PARENT (former ANY_PARENT?) Squid used the first
    considered-alive parent it could reach. This happens when none of the
    specific parent cache selection algorithms (e.g., userhash or carp) were
    enabled, all enabled algorithms failed to find a suitable parent, or all
    suitable parents found by those algorithms failed when Squid tried to
    forward the request to them.
  • INVALID CODE part of src/peer_select.c:hier_strings[].

Almost any of these may be preceded by ‘TIMEOUT_’ if the two-second
(default) timeout occurs waiting for all ICP replies to arrive from
neighbors, see also the icp_query_timeout configuration option.

The following hierarchy codes were removed from Squid-2:

code                  meaning
--------------------  -------------------------------------------------
PARENT_UDP_HIT_OBJ    hit objects are not longer available.
SIBLING_UDP_HIT_OBJ   hit objects are not longer available.
SSL_PARENT_MISS       SSL can now be handled by squid.
FIREWALL_IP_DIRECT    No special logging for hosts inside the firewall.
LOCAL_IP_DIRECT       No special logging for local networks.

🔗 store.log

This file covers the objects currently kept on disk or removed ones. As
a kind of transaction log (or journal) it is usually used for debugging
purposes. A definitive statement, whether an object resides on your
disks is only possible after analyzing the complete log file. The
release (deletion) of an object may be logged at a later time than the
swap out (save to disk).

The store.log file may be of interest to log file analysis which looks
into the objects on your disks and the time they spend there, or how
many times a hot object was accessed. The latter may be covered by
another log file, too. With knowledge of the cache_dir configuration
option, this log file allows for a URL to filename mapping without
recursing your cache disks. However, the Squid developers recommend to
treat store.log primarily as a debug file, and so should you, unless
you know what you are doing.

The print format for a store log entry (one line) consists of thirteen
space-separated columns, compare with the storeLog() function in file
src/store_log.c:

9ld.%03d %-7s %02d %08X %s %4d %9ld %9ld %9ld %s %ld/%ld %s %s
  1. time The timestamp when the line was logged in UTC with a
    millisecond fraction.
  2. action The action the object was sumitted to, compare with
    src/store_log.c:

    • CREATE Seems to be unused.
    • RELEASE The object was removed from the cache (see also
      file number below).
    • SWAPOUT The object was saved to disk.
    • SWAPIN The object existed on disk and was read into memory
  3. dir number The cache_dir number this object was stored into,
    starting at 0 for your first cache_dir line.
  4. file number The file number for the object storage file. Please
    note that the path to this file is calculated according to your
    cache_dir configuration. A file number of FFFFFFFF indicates
    “memory only” objects. Any action code for such a file number refers
    to an object which existed only in memory, not on disk. For
    instance, if a RELEASE code was logged with file number
    FFFFFFFF, the object existed only in memory, and was released from
    memory.
  5. hash The hash value used to index the object in the cache. Squid
    currently uses MD5 for the hash value.
  6. status The HTTP reply status code.
  7. datehdr The value of the HTTP Date reply header.
  8. lastmod The value of the HTTP Last-Modified reply header.
  9. expires The value of the HTTP “Expires: “ reply header.
  10. type The HTTP Content-Type major value, or “unknown” if it
    cannot be determined.
  11. sizes This column consists of two slash separated fields:

    • The advertised content length from the HTTP Content-Length
      reply header.
    • The size actually read.
    • If the advertised (or expected) length is missing, it will
      be set to zero. If the advertised length is not zero, but
      not equal to the real length, the object will be released
      from the cache.
  12. method The request method for the object, e.g. GET.
  13. key The key to the object, usually the URL.

    • The datehdr, lastmod, and expires values are all
      expressed in UTC seconds. The actual values are parsed from the
      HTTP reply headers. An unparsable header is represented by a
      value of -1, and a missing header is represented by a value of
      -2.

🔗 swap.state

This file has a rather unfortunate history which has led to it often
being called the swap log. It is in fact a journal of the cache
index with a record of every cache object written to disk. It is read
when Squid starts up to “reload” the cache quickly.

If you remove this file when squid is NOT running, you will
effectively wipe out your cache index of contents. Squid can rebuild it
from the original files, but that procedure can take a long time as
every file in the cache must be fully scanned for meta data.

If you remove this file while squid IS running, you can easily
recreate it. The safest way is to simply shutdown the running process:

This will disrupt service, but at least you will have your swap log
back. Alternatively, you can tell squid to rotate its log files. This
also causes a clean swap log to be written.

By default the swap.state file is stored in the top-level of each
cache_dir. You can move the logs to a different location with the
cache_swap_state option.

The file is a binary format that includes MD5 checksums, and
StoreEntry fields. Please see the Programmers’ Guide for information
on the contents and format of that file.

🔗 squid.out

If you run your Squid from the RunCache script, a file squid.out
contains the Squid startup times, and also all fatal errors, e.g. as
produced by an assert() failure. If you are not using RunCache, you
will not see such a file.

:warning: RunCache
has been obsoleted since Squid-2.6.
Modern Squid run as daemons usually log this output to the system
syslog facility or if run manually to stdout for the account which
operates the master daemon process.

🔗 useragent.log

:warning:
Starting from
Squid-3.2
this log has become one of the default access.log
formats and is always available for use. It is no longer a special
separate log file.

🔗 Which log files can I delete safely?

You should never delete access.log, store.log, or cache.log while
Squid is running. With Unix, you can delete a file when a process has
the file opened. However, the filesystem space is not reclaimed until
the process closes the file.

If you accidentally delete swap.state while Squid is running, you can
recover it by following the instructions in the previous questions. If
you delete the others while Squid is running, you can not recover them.

The correct way to maintain your log files is with Squid’s “rotate”
feature. You should rotate your log files at least once per day. The
current log files are closed and then renamed with numeric extensions
(.0, .1, etc). If you want to, you can write your own scripts to archive
or remove the old log files. If not, Squid will only keep up to
logfile_rotate
versions of each log file. The logfile rotation procedure also writes a
clean swap.state file, but it does not leave numbered versions of the
old files.

If you set
logfile_rotate
to 0, Squid simply closes and then re-opens the logs. This allows
third-party logfile management systems, such as newsyslog, to maintain
the log files.

To rotate Squid’s logs, simple use this command:

For example, use this cron entry to rotate the logs at midnight:

0 0 * * * /usr/local/squid/bin/squid -k rotate

🔗 How can I disable Squid’s log files?

To disable access.log:

To disable store.log:

To disable cache.log:

:warning:
It is a bad idea to disable the cache.log because this file contains
many important status and debugging messages.
However, if you really want to, you can

:warning:
If /dev/null is specified to any of the above log files,
logfile_rotate MUST
also be set to 0 or else risk Squid rotating away /dev/null making
t a plain log file

:information_source:
Instead of disabling the log files, it is advisable to use a smaller value
for logfile_rotate
and properly rotating Squid’s log files in your cron.
That way, your log files are more controllable and
self-maintained by your system

🔗 What is the maximum size of access.log?

Squid does not impose a size limit on its log files. Some operating
systems have a maximum file size limit, however. If a Squid log file
exceeds the operating system’s size limit, Squid receives a write error
and shuts down. You should regularly rotate Squid’s log files so that
they do not become very large.

:warning:
Logging is very important to Squid.
In fact, it is so important that it will shut itself down if it
can’t write to its logfiles. This includes cases such as a full log disk,
or logfiles getting too big.

🔗 My log files get very big!

You need to rotate your log files with a cron job. For example:

0 0 * * * /usr/local/squid/bin/squid -k rotate

When logging debug information into cache.log it can easily become
extremely large and when a long access.log traffic history is required
(ie by law in some countries) storing large cache.log for that time is
not reasonable. From Squid-3.2
cache.log can be rotated with an individual cap set by
debug_options
rotate=N} option to store fewer of these large files in the .0 to .N
series of backups. The default is to store the same number as with
access.log and set in the
logfile_rotate
directive.

🔗 I want to use another tool to maintain the log files.

If you set
logfile_rotate
to 0, Squid simply closes and then re-opens the logs. This allows
third-party logfile management systems, such as
newsyslog or
logrotate, to maintain the log files.

🔗 Managing log files

The preferred log file for analysis is the access.log file in native
format. For long term evaluations, the log file should be obtained at
regular intervals. Squid offers an easy to use API for rotating log
files, in order that they may be moved (or removed) without disturbing
the cache operations in progress. The procedures were described above.

Depending on the disk space allocated for log file storage, it is
recommended to set up a cron job which rotates the log files every 24,
12, or 8 hour. You will need to set your
logfile_rotate
to a sufficiently large number. During a time of some idleness, you can
safely transfer the log files to your analysis host in one burst.

Before transport, the log files can be compressed during off-peak time.
On the analysis host, the log file are concatenated into one file, so
one file for 24 hours is the yield. Also note that with
log_icp_queries
enabled, you might have around 1 GB of uncompressed log information per
day and busy cache. Look into you cache manager info page to make an
educated guess on the size of your log files.

Some basic advice to obey when handling and processing log files:

  • Respect the privacy of your clients when publishing results.
  • Keep logs unavailable unless anonymized. Most countries have laws on
    privacy protection, and some even on how long you are legally
    allowed to keep certain kinds of information.
  • Rotate and process log files at least once a day. Even if you don’t
    process the log files, they will grow quite large, see My log files
    get very big     above here. If you rely on processing the log files,
    reserve a large enough partition solely for log files.
  • Keep the size in mind when processing. It might take longer to
    process log files than to generate them!
  • Limit yourself to the numbers you are interested in. There is data
    beyond your dreams available in your log file, some quite obvious,
    others by combination of different views. Here are some examples for
    figures to watch:

    • The hosts using your cache.
    • The elapsed time for HTTP requests — this is the latency the
      user sees. Usually, you will want to make a distinction for HITs
      and MISSes and overall times. Also, medians are preferred over
      averages.
    • The requests handled per interval (e.g. second, minute or hour).

🔗 Why do I get ERR_NO_CLIENTS_BIG_OBJ messages so often?

This message means that the requested object was in “Delete Behind” mode
and the user aborted the transfer. An object will go into “Delete
Behind” mode if

  • It is larger than maximum_object_size
  • It is being fetched from a neighbor which has the proxy-only
    option set

🔗 What does ERR_LIFETIME_EXP mean?

This means that a timeout occurred while the object was being
transferred. Most likely the retrieval of this object was very slow (or
it stalled before finishing) and the user aborted the request. However,
depending on your settings for quick_abort, Squid may have continued
to try retrieving the object. Squid imposes a maximum amount of time on
all open sockets, so after some amount of time the stalled request was
aborted and logged win an ERR_LIFETIME_EXP message.

🔗 Retrieving “lost” files from the cache

“I’ve been asked to retrieve an object which was accidentally destroyed
at the source for recovery. So, how do I figure out where the things are
so I can copy them out and strip off the headers?””

The following method applies only to the Squid-1.1 versions:

Use grep to find the named object (URL) in the cache.log file. The
first field in this file is an integer file number.

Then, find the file fileno-to-pathname.pl from the “scripts” directory
of the Squid source distribution. The usage is

perl fileno-to-pathname.pl [-c squid.conf]

file numbers are read on stdin, and pathnames are printed on stdout.

🔗 Can I use store.log to figure out if a response was cachable?

Sort of. You can use store.log to find out if a particular response
was cached.

Cached responses are logged with the SWAPOUT tag. Uncached responses are
logged with the RELEASE tag.

However, your analysis must also consider that when a cached response is
removed from the cache (for example due to cache replacement) it is also
logged in store.log with the RELEASE tag. To differentiate these two,
you can look at the filenumber (3rd) field. When an uncachable response
is released, the filenumber is FFFFFFFF (-1). Any other filenumber
indicates a cached response was released.

🔗 Can I pump the squid access.log directly into a pipe?

Several people have asked for this, usually to feed the log into some
kind of external database, or to analyze them in real-time.

The answer is No. Well, yes, sorta. Using a pipe directly opens up a
whole load of possible problems.

:warning:
Logging is very important to Squid.
In fact, it is so important that it will shut itself down if it
can’t write to its logfiles.

There are several alternatives which are much safer to setup and use.
The basic capabilities present are :

See the Log Modules feature
for technical details on setting up a daemon or other output modules.

To the FAQ Index

Navigation: Site Search,
Site Pages,
Categories, 🔼 go up

Источник

Понравилась статья? Поделить с друзьями:

Интересное по теме:

  • Sql ошибка 258
  • Sql ошибка 209
  • Squad ошибка unreal engine 4 crash reporter
  • Sql ошибка 2003
  • Sqlyog ошибка 2058

    • 0 0 голоса
    Рейтинг статьи
    Подписаться
    Уведомить о
    guest

    0 комментариев
    Старые
    Новые Популярные
    Межтекстовые Отзывы
    Посмотреть все комментарии