Thursday, June 21, 2018

vRealize Log Insight Data Sets using custom filters


IMPORTANT NOTE: All log file examples are not real and exact PCF log files. Since I do not have a running PCF environment while writing this post, I have used some fake examples and modified them to meet my needs.



Let’s consider following use case.

You are responsible for operating a Pivotal Cloud Foundry (PCF) environment.
PCF is sending all log messages for all spaces, orgs and apps etc. to one central vRealize Log Insight cluster.
Your developers using the PCF services would like to access their log messages.
Now, you could just grant read access to all developers but doing that you would allow every developer see all messages received by vRLI, not only PCF related.

The first and pretty easy solution is to leverage the static tag already being used by the PCF Content Pack.

The installation instruction of the content pack says:

Log Insight Setup:
1.      Obtain a Virtual IP from your IT department that will be under the same Subnet as your Log Insight environment.
3.    When prompted for `a list of static tags (key=value)`, enter the following:
          
            product=pcf

This static tag can be used for creation of an appropriated data set, which will contain only logs from your PCF environment.


If you run this settings in interactive analytics you will get only those log messages which are tagged with "product=pcf":


But what if you want your developers to access only logs which belongs to their PCF org or/and space?

Reading the documentation, you could come up with the idea of using extracted fields:

"Use the first drop-down menu to select a field defined within vRealize Log Insight to filter on. 

For example, hostname. 

The list contains all defined fields that are available statically, in content packs, and in custom content."


But if you try to use your own extracted fields in a data set you will notice that this kind of fields are not available in a data set configuration.

The solution I have used with my last customer was to configure the vRLI agent on the PCF syslog server to set static fields "dynamically" via RegEx expression. These static fields based on app or space IDs can be used in data set filters. 

NOTE: This is just an example (fake messages):

[filelog|pcf-syslog]
directory=/var/log/pcf_sample
include=*.log
tags={"tkopton_tag01":"testing-orgs"}
parser=pcforg-parser
enabled=yes

[parser|pcforg-parser]
base_parser=regex
format=cf_org_id=(?<tkopton_tag02>\S+\b)
debug=no
next_parser=pcfspace-parser

[parser|pcfspace-parser]
base_parser=regex
format=cf_space_id="(?<tkopton_tag03>\S+\b)
debug=no
next_parser=syslog_parser



 Now, you can use those fields as filters in data sets, like:



And the test using "Run in Interactive Analytics":


IMPORTANT NOTE: Tested on a PCF Linux syslog server, I do not think, this will work on a Windows system;-)






11 comments:

  1. I appreciate for such a valuable post and also very attractive blog. This is the high-grade content and helped to me. Believe me, your creative style is very super. I am hoping for more unique ideas from your post...
    Corporate Training in Chennai
    Corporate Training institute in Chennai
    Social Media Marketing Courses in Chennai
    Embedded System Course Chennai
    Linux Training in Chennai
    Tableau Training in Chennai
    Corporate Training in Chennai
    Corporate Training institute in Chennai

    ReplyDelete
  2. 6 Reasons Why You Can't Nd10x Review Without Social Media https://www.reviewengin.com/nd10x-review/

    ReplyDelete
  3. How To Buy A Internet Marketing On A Shoestring Budgethttps://www.reviewengin.com/

    ReplyDelete
  4. Unhustled 6 Figure Freedom Review On A Budget: 9 Tips From The Great Depression https://www.reviewengin.com/unhustled-6-figure-freedom-review/

    ReplyDelete
  5. Apply These 8 Secret Techniques To Improve Niche Profit Fast Track https://www.reviewengin.com/niche-profit-fast-track-review/

    ReplyDelete
  6. WebFX will continually monitor the success of your campaign and make changes which are constructive and supportive to your Internet media buying campaign goals. You want to make sure your brand remains consistent over time; that each and every tactic is synchronized to achieve the goals of your marketing campaign. Furthermore, a large portion of a media buyer's job involves networking with clients, business partners and other industry professionals after regular office hours. Queries are fetched immediately from Blue Dart's database, providing a seamless service for the client and his customers, and bringing the entire chain within his domain. All of these steps are essential in helping us to identify where in the funnel a prospect is being touched by your brand and what media channels are making the most impact. From there you can build a local clientele and start hiring others to work for your business. Moving at the speed of today's marketplace, we deliver for our clients the influence and engagement they need in the ever-evolving, multi-screen, multi-channel world of media buying.

    In our recent eBook Marketing Moneyball: Why In-House Advertising Changes Everything , we outline a back-of-the-envelope cost benefit analysis, where a company spending $2,000,000 annually on digital advertising can save $500,000 by taking their advertising in-house. Suddenly, you learn that your biggest competitor has just launched their own spring sundress ad campaign targeting consumers on the same channels. The candle-making business can be initiated from home with low investment of approximately Rs 20,000-Rs 30,000. Founded in 2003, Small Business Trends is an award-winning online publication for small business owners, entrepreneurs and the people who interact with them. Instead, just serve the right ads to the right people through programmatic marketing. All of our rights and obligations under these Business Terms are freely assignable by us to any of our affiliates or in connection with a merger, acquisition, restructuring, or sale of assets, or by operation of law or otherwise, and we may transfer Company's information to any of our affiliates, successor entities, or new owner in connection with such a merger, acquisition, restructuring, or sale of assets.

    Programmatic advertising happens in real time by utilizing algorithms to allow advertisers to bid for the right to place an ad on a web page. Beyond Summits Ltd is a leading all-media China marketing agency, offering one-stop total marketing solutions. You may notice on first inspection that programmatic advertising vs. direct buying have substantial differences between them. Starting up this business demands a small capital investment of around Rs 50,000-Rs 1 lakh. Our consumer-centric media planning approach marries context and creative, ensuring every customer interaction is a match for your customer's desires—while taking advantage of the channels, tactics and customer journey moments with the highest return. The short and simple answer is: you rent media real estate (radio slot, billboard, online banner ad) that will allow you to get your message in front of the right audience. This is a collection of advice from successful professionals in the industry, agencies and analyst groups around the web for media buyers looking to make a splash in their industry. https://www.reviewengin.com/5-simple-media-buying-lessons-to-boost-your-business/

    ReplyDelete
  7. Our custom boxes are manufactured in house by promising care and attention so that client requirements are fulfilled in an effective manner. For assuring an eco-friendly solution, the custom boxes created by BPB are from 100% recycle material to manage a healthy and green environment. For almost all of our products, designing and shipping is definitely Free of cost.
    Hemp Oil Boxes

    Beard Oil Boxes

    Medicine Boxes

    Packaging Boxes Products

    ReplyDelete
  8. Great post mate, thank you for the valuable and useful information. Keep up the good work! FYI, please check this depression, stress, and anxiety

    Silver Vinyl Stickers

    Transparent Stickers

    Kraft Paper Stickers

    Pvc Stickers

    Embossed Paper Stickers

    Custom Stickers Printing

    ReplyDelete