IMPORTANT NOTE: All log file examples
are not real and exact PCF log files. Since I do not have a running PCF
environment while writing this post, I have used some fake examples and
modified them to meet my needs.
Let’s
consider following use case.
You are
responsible for operating a Pivotal Cloud Foundry (PCF) environment.
PCF is
sending all log messages for all spaces, orgs and apps etc. to one central
vRealize Log Insight cluster.
Your
developers using the PCF services would like to access their log messages.
Now, you
could just grant read access to all developers but doing that you would allow
every developer see all messages received by vRLI, not only PCF related.
The first
and pretty easy solution is to leverage the static tag already being used by
the PCF Content Pack.
The
installation instruction of the content pack says:
Log Insight
Setup:
1. Obtain a Virtual IP from your IT
department that will be under the same Subnet as your Log Insight environment.
2. Follow the `Procedure` defined in the
following link: http://pubs.vmware.com/log-insight-33/topic/com.vmware.log-insight.administration.doc/GUID-72CB469A-F276-438D-BC93-77853E2CC0B9.html
3. When prompted for `a list of static
tags (key=value)`, enter the following:
product=pcf
This static tag can be used
for creation of an appropriated data set, which will contain only logs from your
PCF environment.
If you run this settings in interactive analytics you will get only those log messages which are tagged with "product=pcf":
But what if you want your
developers to access only logs which belongs to their PCF org or/and space?
Reading the documentation, you could come up with the idea of using extracted fields:
"Use the first drop-down menu to select a field defined within vRealize Log Insight to filter on.
For example, hostname.
The list contains all defined fields that are available statically, in content packs, and in custom content."
But if you try to use your own extracted fields in a data set you will notice that this
kind of fields are not available in a data set configuration.
The solution I have used with my last customer
was to configure the vRLI agent on the PCF syslog server to set static fields "dynamically" via RegEx expression. These static fields based on app or space IDs can be used in data set filters.
NOTE: This is just an example (fake messages):
[filelog|pcf-syslog]
directory=/var/log/pcf_sample
include=*.log
tags={"tkopton_tag01":"testing-orgs"}
parser=pcforg-parser
enabled=yes
[parser|pcforg-parser]
base_parser=regex
format=cf_org_id=(?<tkopton_tag02>\S+\b)
debug=no
next_parser=pcfspace-parser
[parser|pcfspace-parser]
base_parser=regex
format=cf_space_id="(?<tkopton_tag03>\S+\b)
debug=no
next_parser=syslog_parser
And the test using "Run in Interactive Analytics":
IMPORTANT NOTE: Tested on a PCF Linux syslog server, I do not think, this will work on a Windows system;-)