I like iostat, but...

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

I like iostat, but...

David Wolfskill
... I rather wish I could get the same information via sysctl.  (Well,
something seems to be available via the "opaque" kern.devstat.all
sysctl(8) variable, but sysctl(8) doesn't display all of it, and parsing
it seems as if that would require knowledge about the internals of the
system where the data were acquired.)

If iostat(8) could be taught to (optionally) provide a timestamp, that
might suffice.

The problem I'm trying to solve is this: I need to be able to acquire
various resource counters (along with timestamps), so I can post-process
the acquired data (generally, on a system other than the one where the
data were gathered) in order to be able to see how the
resource-consumption changes over the duration of a moderately
long-running task (typically. 0.5 - 8 hrs.).

I have (with some success) cobbled up a shell script to do much of
this.  (And yes, I've measured the behavior of some typical workloads
in this environment vs. merely running the workload under "/usr/bin/time
-lpo", with 5 test iterations for each, there was no statistically
significant difference with a 95% confidence interval.)

The script:

* Parses its arguments, and from that information constructs a command
  line (which invokes sysctl(8), and (optionally) netstat(1), and
  pipes that output through awk(1)).

* Determines the current time-of-day ("date +%s")

* Enters a loop, which:
  + "eval"s the constructed command line (causing a timestamped line of
    information to be spat out standard output); the timestamp is from
    the most-recently-acquired time-of-day.
  + Determines the current time-of-day ("date +%s").
  + Based on the desired sampling interval, calculates the number of
    seconds to sleep.  (If I could get strftime to format fractional
    seconds, that could be handy.)
  + Sleeps for the calculated interval.
  + Determines the current time-of-day ("date +%s").

And for most things I care about, it works fairly well.

Further, I do this as a shell script precisely so I don't need to build
a new version of the tool for a new target system: the script purposely
only requires tools that are in base FreeBSD, and requires no special
privilege to use.

For some sample graphs I have generated from this kind of data, please
see <http://www.catwhisker.org/~david/FreeBSD/bmake/>.

I believe that having an ability to correlate the "iostat -x"
information with the CPU, load average, and memory utilization would be
useful -- but I don't see a reasonable way to go about it.  If anyone
has suggestions, I'm listening. :-}

Peace,
david
--
David H. Wolfskill [hidden email]
Taliban: Evil cowards with guns afraid of truth from a 14-year old girl.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.

attachment0 (968 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: I like iostat, but...

Andriy Gapon
On 23/09/2014 00:22, David Wolfskill wrote:
> ... I rather wish I could get the same information via sysctl.  (Well,
> something seems to be available via the "opaque" kern.devstat.all
> sysctl(8) variable, but sysctl(8) doesn't display all of it, and parsing
> it seems as if that would require knowledge about the internals of the
> system where the data were acquired.)

Perhaps sysutils/devstat could be of help?

> If iostat(8) could be taught to (optionally) provide a timestamp, that
> might suffice.
>
> The problem I'm trying to solve is this: I need to be able to acquire
> various resource counters (along with timestamps), so I can post-process
> the acquired data (generally, on a system other than the one where the
> data were gathered) in order to be able to see how the
> resource-consumption changes over the duration of a moderately
> long-running task (typically. 0.5 - 8 hrs.).
>
> I have (with some success) cobbled up a shell script to do much of
> this.  (And yes, I've measured the behavior of some typical workloads
> in this environment vs. merely running the workload under "/usr/bin/time
> -lpo", with 5 test iterations for each, there was no statistically
> significant difference with a 95% confidence interval.)
>
> The script:
>
> * Parses its arguments, and from that information constructs a command
>   line (which invokes sysctl(8), and (optionally) netstat(1), and
>   pipes that output through awk(1)).
>
> * Determines the current time-of-day ("date +%s")
>
> * Enters a loop, which:
>   + "eval"s the constructed command line (causing a timestamped line of
>     information to be spat out standard output); the timestamp is from
>     the most-recently-acquired time-of-day.
>   + Determines the current time-of-day ("date +%s").
>   + Based on the desired sampling interval, calculates the number of
>     seconds to sleep.  (If I could get strftime to format fractional
>     seconds, that could be handy.)
>   + Sleeps for the calculated interval.
>   + Determines the current time-of-day ("date +%s").
>
> And for most things I care about, it works fairly well.
>
> Further, I do this as a shell script precisely so I don't need to build
> a new version of the tool for a new target system: the script purposely
> only requires tools that are in base FreeBSD, and requires no special
> privilege to use.
>
> For some sample graphs I have generated from this kind of data, please
> see <http://www.catwhisker.org/~david/FreeBSD/bmake/>.
>
> I believe that having an ability to correlate the "iostat -x"
> information with the CPU, load average, and memory utilization would be
> useful -- but I don't see a reasonable way to go about it.  If anyone
> has suggestions, I'm listening. :-}


--
Andriy Gapon
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: I like iostat, but...

Borja Marcos-2
In reply to this post by David Wolfskill

On Sep 22, 2014, at 11:22 PM, David Wolfskill wrote:

> ... I rather wish I could get the same information via sysctl.  (Well,
> something seems to be available via the "opaque" kern.devstat.all
> sysctl(8) variable, but sysctl(8) doesn't display all of it, and parsing
> it seems as if that would require knowledge about the internals of the
> system where the data were acquired.)

Reading sysctl from a small C program is not hard at all. I did it for devilator (a data recollector for Orca). And
there's a lot of data available. An advantage is, you avoid launching several processes.





Borja.





_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: I like iostat, but...

Stefan Parvu-2
In reply to this post by David Wolfskill

> ... I rather wish I could get the same information via sysctl.  (Well,
> something seems to be available via the "opaque" kern.devstat.all
> sysctl(8) variable, but sysctl(8) doesn't display all of it, and parsing
> it seems as if that would require knowledge about the internals of the
> system where the data were acquired.)

I gave up parsing sysctl via Perl for disks and network devices. It would be
nice to have devstat properly working via sysctl for disk devices. Similar way
kern.cp_times does. Currently there is no simple way to extract per disk stats from
sysctl as a Perl or Sh consumer, unless we build a C module to do that.

> If iostat(8) could be taught to (optionally) provide a timestamp, that
> might suffice.

In fact all performance userland tools should be able to nicely produce timestamp
CSV records which can be used for capacity planning and feed them to an
analytic product. Something like:

1411445589:4.01:16.03:383.97:1.93:0.29:1.68:0.11:95.99:0.00:0.00:0.00:0.00:123.00:229516.00:570992.00

or something like iostat:

1411445733:ada0:0.00:0.00:0.00:0.00:0.00:0.00:0.00

where the first field would be always the timestmp, unix time. It is not that complicated
but it does not exist.
 
> The problem I'm trying to solve is this: I need to be able to acquire
> various resource counters (along with timestamps), so I can post-process
> the acquired data (generally, on a system other than the one where the
> data were gathered) in order to be able to see how the
> resource-consumption changes over the duration of a moderately
> long-running task (typically. 0.5 - 8 hrs.).

My idea of having standard data recorders: sysrec, cpurec, nicrec diskrec
which can extract: overall system consumption, per device statistics.
http://www.systemdatarecorder.org/recording/agents.html 

First place to start with will be sysrec, the main recorder which will report overall
system consumption:

  * cpu utilization across all CPUs
  * memory utilization
  * disk IO across all disk devices
  * network IO across all NICs
  * LA
http://www.systemdatarecorder.org/recording/sysrec_freebsd.html


> I believe that having an ability to correlate the "iostat -x"
> information with the CPU, load average, and memory utilization would be
> useful -- but I don't see a reasonable way to go about it.  If anyone
> has suggestions, I'm listening. :-}

How about sysrec, like describe above. I dont like this version because it is using
iostat, netstat for disks and NICs but I dont have a better solution right now.

There is another way for NIC stats to use libstatgrab and a Perl module for it. I got some
troubles in using it under FreeBSD 10 and having some considerable memory
usage over time, and I did not have time to test it further. But I will a bit later.
There are some limitations about libstatgrab: does not know per CPU stats for
example.


http://www.i-scream.org/libstatgrab/

--
Stefan Parvu <[hidden email]>
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: I like iostat, but...

Borja Marcos-2

On Sep 23, 2014, at 10:38 AM, Stefan Parvu wrote:

>
>> ... I rather wish I could get the same information via sysctl.  (Well,
>> something seems to be available via the "opaque" kern.devstat.all
>> sysctl(8) variable, but sysctl(8) doesn't display all of it, and parsing
>> it seems as if that would require knowledge about the internals of the
>> system where the data were acquired.)
>
> I gave up parsing sysctl via Perl for disks and network devices. It would be
> nice to have devstat properly working via sysctl for disk devices. Similar way
> kern.cp_times does. Currently there is no simple way to extract per disk stats from
> sysctl as a Perl or Sh consumer, unless we build a C module to do that.

Anyway, for disk stats GEOM offers a nice API. You can get delays per GEOM provider, bandwidths, etc.





Borja.

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: I like iostat, but...

Julian Elischer-5
In reply to this post by Stefan Parvu-2
On 9/23/14, 4:38 PM, Stefan Parvu wrote:

>> ... I rather wish I could get the same information via sysctl.  (Well,
>> something seems to be available via the "opaque" kern.devstat.all
>> sysctl(8) variable, but sysctl(8) doesn't display all of it, and parsing
>> it seems as if that would require knowledge about the internals of the
>> system where the data were acquired.)
> I gave up parsing sysctl via Perl for disks and network devices. It would be
> nice to have devstat properly working via sysctl for disk devices. Similar way
> kern.cp_times does. Currently there is no simple way to extract per disk stats from
> sysctl as a Perl or Sh consumer, unless we build a C module to do that.
>
>> If iostat(8) could be taught to (optionally) provide a timestamp, that
>> might suffice.
> In fact all performance userland tools should be able to nicely produce timestamp
> CSV records which can be used for capacity planning and feed them to an
> analytic product. Something like:
>
> 1411445589:4.01:16.03:383.97:1.93:0.29:1.68:0.11:95.99:0.00:0.00:0.00:0.00:123.00:229516.00:570992.00
>
> or something like iostat:
>
> 1411445733:ada0:0.00:0.00:0.00:0.00:0.00:0.00:0.00
>
> where the first field would be always the timestmp, unix time. It is not that complicated
> but it does not exist.
>  
>> The problem I'm trying to solve is this: I need to be able to acquire
>> various resource counters (along with timestamps), so I can post-process
>> the acquired data (generally, on a system other than the one where the
>> data were gathered) in order to be able to see how the
>> resource-consumption changes over the duration of a moderately
>> long-running task (typically. 0.5 - 8 hrs.).
> My idea of having standard data recorders: sysrec, cpurec, nicrec diskrec
> which can extract: overall system consumption, per device statistics.
> http://www.systemdatarecorder.org/recording/agents.html
>
> First place to start with will be sysrec, the main recorder which will report overall
> system consumption:
>
>    * cpu utilization across all CPUs
>    * memory utilization
>    * disk IO across all disk devices
>    * network IO across all NICs
>    * LA
> http://www.systemdatarecorder.org/recording/sysrec_freebsd.html
>
>
>> I believe that having an ability to correlate the "iostat -x"
>> information with the CPU, load average, and memory utilization would be
>> useful -- but I don't see a reasonable way to go about it.  If anyone
>> has suggestions, I'm listening. :-}
> How about sysrec, like describe above. I dont like this version because it is using
> iostat, netstat for disks and NICs but I dont have a better solution right now.
>
> There is another way for NIC stats to use libstatgrab and a Perl module for it. I got some
> troubles in using it under FreeBSD 10 and having some considerable memory
> usage over time, and I did not have time to test it further. But I will a bit later.
> There are some limitations about libstatgrab: does not know per CPU stats for
> example.
>
>
> http://www.i-scream.org/libstatgrab/
>
try this:
http://lists.freebsd.org/pipermail/freebsd-current/2006-August/065003.html

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: I like iostat, but...

Stefan Parvu-2
In reply to this post by Borja Marcos-2

> Anyway, for disk stats GEOM offers a nice API. You can get delays per GEOM provider, bandwidths, etc.

Are you talking about C consumers or I can do that using Perl, Sh ?

Is there any way to consume the metrics via Perl, for example ? I could not find
any decent documentation how to do that except some emails:
http://lists.freebsd.org/pipermail/freebsd-questions/2009-March/194081.html


Any ideas ?

Thanks,
--
Stefan Parvu <[hidden email]>
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: I like iostat, but...

Borja Marcos-2

On Sep 23, 2014, at 11:19 AM, Stefan Parvu wrote:

>
>> Anyway, for disk stats GEOM offers a nice API. You can get delays per GEOM provider, bandwidths, etc.
>
> Are you talking about C consumers or I can do that using Perl, Sh ?
>
> Is there any way to consume the metrics via Perl, for example ? I could not find
> any decent documentation how to do that except some emails:
> http://lists.freebsd.org/pipermail/freebsd-questions/2009-March/194081.html

C consumers, although of course you can write a Perl module in C in case you prefer Perl.

If the shock of reading atrocious code won't make you pull your eyes out of your sockets I
can send you the latest version of devilator a FreeBSD specific data collector for Orca
(https://www.orcaware.com/orca/).

It fetches several system stats (CPU usage, memory usage, etc) and creates text files
with a timestamp. Orca generates RRD databases and graphs consuming that data.

The code is ugly as hell (it's always been an evolving hack) but it works, I've been using it
for years. Maybe it will help to do your own version if you wish.




Borja.

_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: I like iostat, but...

Stefan Parvu-2

> If the shock of reading atrocious code won't make you pull your eyes out of your sockets I
> can send you the latest version of devilator a FreeBSD specific data collector for Orca
> (https://www.orcaware.com/orca/).

sure. Im still thinking how to improve things on my side. Im not C fluent but I can handle
somehow.

> It fetches several system stats (CPU usage, memory usage, etc) and creates text files
> with a timestamp. Orca generates RRD databases and graphs consuming that data.

Sweet. I need to check this. Many thanks indeed.

--
Stefan Parvu <[hidden email]>
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: I like iostat, but...

David Wolfskill
In reply to this post by Julian Elischer-5
On Tue, Sep 23, 2014 at 12:29:22AM +0300, Andriy Gapon wrote:
> On 23/09/2014 00:22, David Wolfskill wrote:
> > ... I rather wish I could get the same information via sysctl.  (Well,
> > something seems to be available via the "opaque" kern.devstat.all
> > sysctl(8) variable, but sysctl(8) doesn't display all of it, and parsing
> > it seems as if that would require knowledge about the internals of the
> > system where the data were acquired.)
>
> Perhaps sysutils/devstat could be of help?
> ...

On Tue, Sep 23, 2014 at 08:56:54AM +0200, Borja Marcos wrote:
> ...
> Reading sysctl from a small C program is not hard at all. I did it for devilator (a data recollector for Orca). And
> there's a lot of data available. An advantage is, you avoid launching several processes.
> ...

On Tue, Sep 23, 2014 at 10:45:14AM +0200, Borja Marcos wrote:
> ...
> Anyway, for disk stats GEOM offers a nice API. You can get delays per GEOM provider, bandwidths, etc.

On Tue, Sep 23, 2014 at 11:38:44AM +0300, Stefan Parvu wrote:
> ...
> I gave up parsing sysctl via Perl for disks and network devices. It would be
> nice to have devstat properly working via sysctl for disk devices. Similar way
> kern.cp_times does. Currently there is no simple way to extract per disk stats from
> sysctl as a Perl or Sh consumer, unless we build a C module to do that.


Folks, I appreciate the suggestions, but they address problems other
than the one I am trying to solve.

In particular:
* I require that the tool must only depend on components of base FreeBSD;
  thus, I don't need to perturb the system I want to measure by
  installing otherwise unneeded software on it.

* I am using a shell script (which uses date, sysctl, netstat, and awk)
  so I don't need to recompile my data acquisition tool.

* The tasks I am trying to measure are software builds similar to
  a stable/10 "make universe" -- but about 2 - 3 times the duration
  (and reading and writing a significantly larger amount of data).

  Thus, the number of additional processes caused by my probe firing
  even as often as once/second is lost in the noise.



> ...
> > If iostat(8) could be taught to (optionally) provide a timestamp, that
> > might suffice.
>
> In fact all performance userland tools should be able to nicely produce timestamp
> CSV records which can be used for capacity planning and feed them to an
> analytic product. Something like:
>
> 1411445589:4.01:16.03:383.97:1.93:0.29:1.68:0.11:95.99:0.00:0.00:0.00:0.00:123.00:229516.00:570992.00
>
> or something like iostat:
>
> 1411445733:ada0:0.00:0.00:0.00:0.00:0.00:0.00:0.00
>
> where the first field would be always the timestmp, unix time. It is not that complicated
> but it does not exist.

The output of my shell script may be described as:

#   <record>    ::= <fieldlist> newline
#   <fieldlist> ::= <field> | <field> tab <fieldlist>
#   <field>     ::= <tag> : <value>
#   <tag>       ::= [_0-9a-zA-Z][-_.0-9a-zA-Z]+
#   <value>     ::= [^\t\n]*

# Each record will have a field with a tag called "time"; the
# associated value will be the number of seconds since the Epoch,
# but will be coerced as necessary to ensure that it is monotonically
# increasing.

# Note that the colon (":") is a valid character in the value part
# of a field (but not in the tag).  Further, there is no whitespace
# on either side of that delimiting colon: everything to the left of the
# colon (up to, but not including, the previous tab, if any) is part of the
# tag; everything to the right of the colon (up to, but not including,
# the next tab or newline) is part of the value.

A prior version of the script output CSV; for this version, I chose to
use the above format because I had situations where some fields showed
up on some lines, but not on others.  That tends to make the use of CSV
a bit problematic.  (On a machine where I post-process the collected
data, I have some Perl that reads the above format, creates new field
names as necessary to cope with multivariate data (e.g., kern.cp_time or
vm.loadavg), and generates CSV from the result.)

> ...
> My idea of having standard data recorders: sysrec, cpurec, nicrec diskrec
> which can extract: overall system consumption, per device statistics.
> http://www.systemdatarecorder.org/recording/agents.html 
> ...

That looks interesting and useful for a broad class of similar problems.

However, as far as I can tell, it is not suitable for the problem(s) I
am trying to solve in this case.

Basically, I have something that works "well enough" for things
like CPU counters, memory usage (at the rather coarse granularity
that top(1) provides, vs. "vmstat -m" output), load avergaes, and
NIC counters, and is readily extensible to any univariate (or simple
list of multivariate) (non-opaque) sysctl OIDs.  I'd like to be
able to include information from the I/O subsystem -- in particular,
data that is accessible from "iostat -x".

Peace,
david
--
David H. Wolfskill [hidden email]
Taliban: Evil cowards with guns afraid of truth from a 14-year old girl.

See http://www.catwhisker.org/~david/publickey.gpg for my public key.

attachment0 (968 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: I like iostat, but...

Borja Marcos




> On 24/9/2014, at 17:09, David Wolfskill <[hidden email]> wrote:
>
>
>> On Tue, Sep 23, 2014 at 10:45:14AM +0200, Borja Marcos wrote:
>> ...
>> Anyway, for disk stats GEOM offers a nice API. You can get delays per GEOM provider, bandwidths, etc.
>
>>
>
>
> Folks, I appreciate the suggestions, but they address problems other
> than the one I am trying to solve.
>
> In particular:
> * I require that the tool must only depend on components of base FreeBSD;
>  thus, I don't need to perturb the system I want to measure by
>  installing otherwise unneeded software on it.


Devilator has no dependencies. It reads sysctl and geom.


>
> Basically, I have something that works "well enough" for things
> like CPU counters, memory usage (at the rather coarse granularity
> that top(1) provides, vs. "vmstat -m" output), load avergaes, and
> NIC counters, and is readily extensible to any univariate (or simple
> list of multivariate) (non-opaque) sysctl OIDs.  I'd like to be
> able to include information from the I/O subsystem -- in particular,
> data that is accessible from "iostat -x".

Check the diskbw.c module. Actually Most of it is borrowed from gstat(8). Just format the output  data as you wish ;)

But you don't need Orca. The agent just creates text files.

Cheers,




Borja.
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"
Reply | Threaded
Open this post in threaded view
|

Re: I like iostat, but...

Alfred Perlstein-2
In reply to this post by David Wolfskill

On 9/24/14 8:09 AM, David Wolfskill wrote:

> On Tue, Sep 23, 2014 at 12:29:22AM +0300, Andriy Gapon wrote:
>> On 23/09/2014 00:22, David Wolfskill wrote:
>>> ... I rather wish I could get the same information via sysctl.  (Well,
>>> something seems to be available via the "opaque" kern.devstat.all
>>> sysctl(8) variable, but sysctl(8) doesn't display all of it, and parsing
>>> it seems as if that would require knowledge about the internals of the
>>> system where the data were acquired.)
>> Perhaps sysutils/devstat could be of help?
>> ...
> On Tue, Sep 23, 2014 at 08:56:54AM +0200, Borja Marcos wrote:
>> ...
>> Reading sysctl from a small C program is not hard at all. I did it for devilator (a data recollector for Orca). And
>> there's a lot of data available. An advantage is, you avoid launching several processes.
>> ...
> On Tue, Sep 23, 2014 at 10:45:14AM +0200, Borja Marcos wrote:
>> ...
>> Anyway, for disk stats GEOM offers a nice API. You can get delays per GEOM provider, bandwidths, etc.
> On Tue, Sep 23, 2014 at 11:38:44AM +0300, Stefan Parvu wrote:
>> ...
>> I gave up parsing sysctl via Perl for disks and network devices. It would be
>> nice to have devstat properly working via sysctl for disk devices. Similar way
>> kern.cp_times does. Currently there is no simple way to extract per disk stats from
>> sysctl as a Perl or Sh consumer, unless we build a C module to do that.
>
> Folks, I appreciate the suggestions, but they address problems other
> than the one I am trying to solve.
>
> In particular:
> * I require that the tool must only depend on components of base FreeBSD;
>    thus, I don't need to perturb the system I want to measure by
>    installing otherwise unneeded software on it.
>
> * I am using a shell script (which uses date, sysctl, netstat, and awk)
>    so I don't need to recompile my data acquisition tool.
>
> * The tasks I am trying to measure are software builds similar to
>    a stable/10 "make universe" -- but about 2 - 3 times the duration
>    (and reading and writing a significantly larger amount of data).
>
>    Thus, the number of additional processes caused by my probe firing
>    even as often as once/second is lost in the noise.
>
>
>
>> ...
>>> If iostat(8) could be taught to (optionally) provide a timestamp, that
>>> might suffice.
>> In fact all performance userland tools should be able to nicely produce timestamp
>> CSV records which can be used for capacity planning and feed them to an
>> analytic product. Something like:
>>
>> 1411445589:4.01:16.03:383.97:1.93:0.29:1.68:0.11:95.99:0.00:0.00:0.00:0.00:123.00:229516.00:570992.00
>>
>> or something like iostat:
>>
>> 1411445733:ada0:0.00:0.00:0.00:0.00:0.00:0.00:0.00
>>
>> where the first field would be always the timestmp, unix time. It is not that complicated
>> but it does not exist.
>
> The output of my shell script may be described as:
>
> #   <record>    ::= <fieldlist> newline
> #   <fieldlist> ::= <field> | <field> tab <fieldlist>
> #   <field>     ::= <tag> : <value>
> #   <tag>       ::= [_0-9a-zA-Z][-_.0-9a-zA-Z]+
> #   <value>     ::= [^\t\n]*
>
> # Each record will have a field with a tag called "time"; the
> # associated value will be the number of seconds since the Epoch,
> # but will be coerced as necessary to ensure that it is monotonically
> # increasing.
>
> # Note that the colon (":") is a valid character in the value part
> # of a field (but not in the tag).  Further, there is no whitespace
> # on either side of that delimiting colon: everything to the left of the
> # colon (up to, but not including, the previous tab, if any) is part of the
> # tag; everything to the right of the colon (up to, but not including,
> # the next tab or newline) is part of the value.
>
> A prior version of the script output CSV; for this version, I chose to
> use the above format because I had situations where some fields showed
> up on some lines, but not on others.  That tends to make the use of CSV
> a bit problematic.  (On a machine where I post-process the collected
> data, I have some Perl that reads the above format, creates new field
> names as necessary to cope with multivariate data (e.g., kern.cp_time or
> vm.loadavg), and generates CSV from the result.)
>
>> ...
>> My idea of having standard data recorders: sysrec, cpurec, nicrec diskrec
>> which can extract: overall system consumption, per device statistics.
>> http://www.systemdatarecorder.org/recording/agents.html
>> ...
> That looks interesting and useful for a broad class of similar problems.
>
> However, as far as I can tell, it is not suitable for the problem(s) I
> am trying to solve in this case.
>
> Basically, I have something that works "well enough" for things
> like CPU counters, memory usage (at the rather coarse granularity
> that top(1) provides, vs. "vmstat -m" output), load avergaes, and
> NIC counters, and is readily extensible to any univariate (or simple
> list of multivariate) (non-opaque) sysctl OIDs.  I'd like to be
> able to include information from the I/O subsystem -- in particular,
> data that is accessible from "iostat -x".
>
David:

check this out:

https://github.com/alfredperlstein/eagleeye

it makes time series data out of a number of tools, it's not really up
to date, but you can probably pull useful code from it.

Also there was the "machine readable output from utilities" GSoC project
that I still need to merge, but that is a larger project.

-Alfred
_______________________________________________
[hidden email] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[hidden email]"