Subject: [Syslog] Doubts on definitions

From: "David B Harrington" <[email-masked]>
Date: Wed, 10 Jan 2007 15:53:05 -0500

Hi,

As we have this discussion, I find that the terminology (and the
architecture it describes) in the protocol document is very ambiguous.
For example,

> > >   The following definitions are used in this document:
> > >   o  An application that can generate a syslog message is called
a
> > >      "sender".

So I have an application (or device) that sends a message via
inter-process
communication to syslogd which then takes the information and formats
it into a syslog message and sends the message into the network. So is
the application the 'sender" or is the syslogd process the sender? Is
the application the originator or is the syslogd process the
originator of the syslog message?

We are dealing with Internet standards, so we should only care about
the process that sends the message onto the (IP) network, not about
the IPC communications - we should be concerned with the on-the-wire
formats and behaviors, not the internal formats and behaviors. 

What if the "application" is a limited-capability-device, such as an
IP-Phone, that sends (over a communications connection) syslog message

content to a host/server that uses a syslogd to format and send a
syslog message across the Internet? Does the syslog device send a
"syslog message" to the server? If so, then is the device the sender
or the originator? And what role does the syslogd perform in this
configuration? Is the syslogd actually a
relay in this case?

I don't think the -protocol- document does a good job of explaining
these relationships, especially describing how a syslogd fits into the
architecture.

Add to that the Windows approach where a syslogd (or a Windows syslog
service) is not provided by the OS, so each application has to provide
its own syslog stack, and the "architecture" gets really hard to model
in an unambiguous manner.

So I understand Glenn's difficulty developing a data
model using the terminology and architecture from the protocol
document. I think Glenn and Tom have very different views of what
architecture needs to be modeled, and the terminology in the
-protocol- document is really not very helpful. 

Remember that Miao also had a problem trying to describe the 
funtionality in the TLS document using the 
sender/receiver/relay/collector terminology.

I personally dislike the "collector" terminology, because the
difference between a reciever and a collector is that a collector
stores the data after receiving the message from the network. We
should
be dealing with the sending and receiving of messages over IP, and it
should be immaterial whether the receiver stores the data (thereby
becoming a receiver-only, aka a collector) or passes it on (thereby
becoming a receiver plus sender, aka a relay). What happens once the
message is off the wire should not matter to the protocol. (It may
matter to -sign- but that is a different issue. It should not matter
to the protocol.)

I think the terminology and architecture are woefully inadequate to
describe in a useful way the various configurations that can and do
occur in existing deployments, and how they relate to on-the-wire
message formats and security.

I can understand why Glenn and Miao preferred to not use the
terminology from protocol-, and I have to wonder if this WG has met
the requirement for a Proposed Standard - "A Proposed Standard
specification is generally stable, has resolved known design choices,
is believed to be well-understood, has received significant community
review, and appears to enjoy enough community interest to be
considered valuable."

It appears that even amongst the people of this WG, the terminology
and architecture are not well-understood. What can we expect when
people who did not help write the specification try to understand,
implement, and use it?

I usually expect a specification being promoted to Proposed Standard
to be clear and unambiguous, and I don't see that level of document
maturity here.

David Harrington
[email-masked] 
[email-masked]
[email-masked]

_______________________________________________
Syslog mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/syslog

---

From: "Rainer Gerhards" <[email-masked]>
Date: Wed, 10 Jan 2007 22:17:19 +0100

David, 

> -----Original Message-----
> From: David B Harrington [mailto:[email-masked]] 
> Sent: Wednesday, January 10, 2007 9:53 PM
> To: [email-masked]
> Subject: [Syslog] Doubts on definitions
> 
> Hi,
> 

Should it really describe all these relationships and how a specific
(though important) sofware architecture works (syslogd)? I thought we
are talking about on-the-wire standards. If we begin to describe how to
implement the inner workings of the local syslog subsystem, we need to
go far beyond what happens on the wire. The, I think, the next step
needed would be to talk to the POSIX folks and ask them to cooperate on
redefining the POSIX API. Next, we need to try to make this an universal
standard. Is this really what we intend to do? I think we should stick
with how different syslog systems can interoperate with each other and
not try to force everyone to use the same SOFTWARE architecture...
 
> in an unambiguous manner.

I am probably not knowledgable enough - but what happens if different
applications based on NET-SNMP run on a single machine? Are they all
connecting back to a single engine and are they required to have the
exact same software architecture and APIs?

> matter to -sign- but that is a different issue. It should not matter

.. Nor should matter how it got on the wire..

> to be clear and unambiguous, and I don't see that level of document

IMHO, the core problem is that we still do not differentiate between the
software - most thinking syslogd - and the *protocol* syslog. It is NOT.
If it were, the IETF would be the wrong place to standardize it. POSIX
would be. I still find that the protocol definitions are clear. I agree
it is not unambigous how to implement different parts, but that is not
because of the on-the-wire architecture but because of existing API sets
(and software architecture). 

I personally would not like to take up the challenge of standardizing
the API set. I do not even find it desirable (from a software
monoculture point of view).

Rainer

---

From: "David B Harrington" <[email-masked]>
Date: Thu, 11 Jan 2007 20:06:10 -0500

Hi Rainer,

Let me explain the problem I am facing as co-chair.
I don't know syslog very well.
I have lots of experience chairing, but little experience with syslog.

I have a lot of experience in writing MIBs, and rule #1 is "you need
to know what you are trying to model before you can model it in a
MIB." But the WG does not seem to have consensus on what it is that we
need to model.

Glenn designed a MIB module that appears more complex than is needed.
He models multiple entities in a system. I don't know what "entity" he
is referring to. Maybe he is talking about multiple applications on a
Windows box, each with its own syslog stack. Maybe he is talking about
multiple software applications that all go through the same daemon.
Maybe he is trying to cover both using the same model. I am trying to
understand what Glenn has modeled and why, but I don't.

Only one person has provided a review of the mib document from the
syslog standpoint - Tom. But Tom talks about syslogd and devices, and
he doesn't seem to have knowledge of non-UNIX configurations. And I
cannot understand how Tom's model maps to the terminology and
architecture of the protocol doc or to Glenn's mib.

When we have only two people, and they have totally different
understandings of what we need to model, we cannot very well achieve
consensus.

As co-chairs, Chris and I need to try to understand what each is
saying, and where the disconnect exists. Unfortunately, the
terminology and architectural concepts are so muddled, I cannot tell
what each person is saying clearly. We have not provided unambiguous
terminology that allows us to clarify the discussion. When  Tom says
application and Glenn says application, I cannot tell if they are
referring to the same thing or totally different concepts. But
"application" is at the core of all our definitions.

Maybe we don't need all the complexity of Glenn's MIB. Maybe a MIB
like the one Tom asked for would be fully adequate for fault,
performance, and configuration monitoring. I just don't know. If we
define a simple MIB that only works for some use cases but not others,
then we lose interoperability if we have to define another MIB for
different use cases. 

*********
Additional syslog voices would be tremendously helpful in determining
this.
*********

Have YOU reviewed the mib document? 
Is the complex design that Glenn used appropriate?
Is the simple design that Tom would prefer enough?
Would the counters in Glenn's MIB be useful to understand the
operation of the two syslog implementations that you work on?
Would the configuration information in Glenn's MIB be useful to
understand the operation of the two syslog implementations that you
work on?

How about the other implementers? How appropriate is Glenn's mib to
model your product? How appropriate is Tom's? What needs to be modeled
to make your product manageable in a vendor-neutral way?

dbh

> From: Rainer Gerhards [mailto:[email-masked]] 
> Sent: Wednesday, January 10, 2007 4:17 PM
> To: David B Harrington; [email-masked]
> Subject: RE: [Syslog] Doubts on definitions
> 
> David, 
>

---

From: "Rainer Gerhards" <[email-masked]>
Date: Fri, 12 Jan 2007 10:17:51 +0100

David,

thanks for your very clear and helpful message.

Find comments inline below.

Rainer

> Sent: Friday, January 12, 2007 2:06 AM
> To: Rainer Gerhards; [email-masked]
> Cc: 'Chris Lonvick'

Let me first say that I am glad to have such an experienced co-chair in
this WG. I really appreciate how you help us to reach our goals.

> MIB." But the WG does not seem to have consensus on what it is that we

This is probably the problem I am facing. I have little experience with
MIBs and I have to admit that I have not even read all SNMP-related
RFCs. It's on my todo list for quite a while, but it is obviously very
time-consuming.

I do not fully understand how SNMP models the *applications* (computer
programs implementing SNMP) running on the various machines. Does SNMP
definitely describe the software architecture that MUST be implemented
on a system? Does it do this even though it has nothing to do with
on-the-wire communications? Even if so -- my understanding and (limited)
practical experience is that at least in the Windows world SNMP
implementations are not much different than syslog. If you install more
than one SNMP capable application, you need to use different
(non-standard) ports, because there is NO OS SNMP engine that everyone
talks to via an API. Instead, each applications brings ist own SNMP
engine with it (well.. at least many do - I do not know one that
doesn't, but I do not know everything). So, as far as *I* know it is
pretty much the same as with syslog where each application brings its
own stack. I have not the slightes idea how SNMP is implemented on *nix.

If I look at any other protocol, it is always the same thing: the stack
is part of an application (think http, SMTP, POP3, IMAP, DNS...). If you
have multiple applications on a single machine, you have multiple stacks
and need to do something about ports. I don't see that syslog is
specific here. This is why -protocol uses the term "application" and I
thought that would be a precise definition.

The only thing that is special with syslog is that under one operating
system (*nix), there is a different architecture with syslogd. It's not
Windows that is different. It is the *nix implementation (at least in my
point of view). The problem is that *nix is obviously the dominant
implementation and that many boxes use linux as "firmware". So this is
probably the root cause of the problem.

Of course it is possible to define the two possible *software
architectures* of syslog. But is this really something that must go into
a RFC? Isn't that way beyond the scope of the IETF?

While thinking about this, I thought that SMTP has defined a software
architecture with MTA und MUA. But even this definition seems not to be
unambigous. See RFC 2821:

##
   However, while these
   terms ### MTA/MUA ### are used with at least the appearance of great
precision in
   other environments, the implied boundaries between MUAs and MTAs
   often do not accurately match common, and conforming, practices with
   Internet mail.  Hence, the reader should be cautious about inferring
   the strong relationships and responsibilities that might be implied
   if these terms were used elsewhere.
##

If we define a syslog message transfer agent and a syslog user agent
(whatever these beasts might be), probably the very same comment would
apply.

OK, back to my honest core question: is the *software architecture*
something we need/can define in a RFC?

> Windows box, each with its own syslog stack. 

If they don't share a syslog stack they also do not share a SNMP stack.
I do not know a single one who currently does this. Remember that there
are neither APIs for syslog NOR for SNMP on Windows.

> Maybe he is talking about

In my view, "application" is simply a piece of software that performs
some task(s). I can not say "process", because it may consist of
multiple processes. Examples are popular SMTP MTAs (Postfix, Exchange,
...) - in syslog, at least SDSC syslog and rsyslog use multiple
processes to implement the "syslog application".

If I try to further define application, I end up with a definition that
is very broad by its very nature:

"A [syslog] application is a computer program that sends or receives
syslog messages."

[with a "computer program" being "... a collection of instructions that
describe a task, or set of tasks, to be carried out by a computer."
(Wikipedia)]. Though the definition is brief, it excludes all process
that talk just via APIs with the syslogd. "Send" and "recieve" require
network activity. So could the above definition eventually be useful? I
first doubted that but begin to see some usefulness in it ... after
re-reading it multiple time.

I can offer to create a paper on the architecture of syslog in different
environments. I am still doubtful if such a description belongs into a
RFC. Maybe it is useful as an informational RFC. But again I think we
can not *standardize* a software architecture. That does not make sense.
Different environments and different use cases require different
architectures.

> Have YOU reviewed the mib document?

Yes, but I have to admit that I did not get a clear impression from the
text. This is not Glenn's fault. The reason is that I do not know enough
about SNMP and I am too unexperienced reviewing MIBS. I have not
provided feedback (except the architecutural run-down I posted) because
I think I can not provide useful in-depth advise without really
understanding the depths of the document.

> Is the simple design that Tom would prefer enough?

I tend to say that the simple design is good enough. On the other hand,
I can envison environments where multiple engines are used inside the
MIB. Even rsyslog could be modelled with a RFC 3195 engine and a "rest
of the syslog world" engine. Actually, the software *has* two stacks.
The same goes for SDSC syslog as well as our MonitorWare products on
Windows. 3195 is so complex, that it doesn't fit in the regular syslog
process. It also requires very different supporting libraries, which
simply do not support the rest of the syslog world's protocol's (I am
talking for all three implementations here because I know some details
of SDSC syslog and have talked a lot with the authors). Thus, it is
necessary from a software architecture / economy point of view to run
two stacks and bring the messages together waaay up on the application
layer. In an ideal world, the architecture would be different. But we
are not in an ideal world...

> understand the operation of the two syslog implementations that you

It provides at least a common ground. The idea of just providing a
configuration file for most of the parameters is extremely useful. But
other than that, there is very limited configuration information in it.
And this is good because existing applications are extremely different -
as (naturally) are there configuration parameters. The real value comes
from the config file, which needs to be fetched separately. So the value
I see in the MIB is that it enables operators to notify a (potentially
large) number of syslog "applications" that they should pull a new set
of configuration files. Given the potentially low number of receivers
(servers) even in large environments I am doubtful if that capability is
really so useful. To be honest, most installations I know already do
this kind of deployment via SSH-based scripts (sounds like the idea why
NetConf was created). I will definitely not start to implement the MIB
module immediately but wait for other implementations and/or customer
demand. I've implemented 3195, and so far this sounds like a big waste
of time. 3195 looked much more useful than the MIB effort. I am sorry to
say that, but nobody (customer, peer, open source community member, log
mailing lists members) has ever asked my about a syslog MIB while I am
often asked for the things that 3195 promises...

The counters are probably more useful. I've seen requests for
performance counters and most implementations offer them (most notabe
exception is stock syslogd and its derivatives [including rsyslog]).

In our Windows application, I would offer the counters just as an
optional, alternate way to retrieve performance counters. The issue here
is that in order to provide the counters via SNMP, we would need to
integrate AND start up a SNMP stack. That would mean potential conflicts
with other SNMP stacks on the same system (as outlined above). That is
not worth the hassle *just* to provide a few performance counters. We
would probably implement it anyhow, as we need to start up an SNMP stack
in the case that the product should also receive SNMP traps. But the MIB
counters would be turend off by default and I suspect few customers
would ever turn them on (because there are other (OS standard) ways of
obtaining them under Windows).

On *nix, it depens on how SNMP is implemented there (which I do not yet
know). If there is "just" an API I can use to register with a SNMP
subsystem, I will probably implement it. If I need to integrate a stack
into my application there, too, I will definitely not do this for just
the counters (except, of course, there is actual demand for it).

I hope these comments are useful.

Rainer

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 12 Jan 2007 10:48:24 +0100

> probably the root cause of the problem.

I like to note that syslog was invented on Unix systems and hence I do
consider the Unix way of doing syslog to be somehow authoritative. But
we do not have to agree on this. ;-)
 
> Different environments and different use cases require different

Dave Harrington's point is that you can't write a proper management
interface (a MIB module) without first understanding the architecture
of the thing you model in the management interface. And this is why
syslog people have to help the SNMP people to get their MIB modules
right. You are not asked to become an expert in SNMP/MIBs - all that
is needed is that you can help draw up a model for a management
interface that matches to ideally all existing syslog implementations.
For that, such a paper would be indeed useful.

/js

-- 
Juergen Schoenwaelder		 {International|Jacobs} University Bremen
<http://www.eecs.iu-bremen.de/>	 P.O. Box 750 561, 28725 Bremen, Germany

---

From: "Rainer Gerhards" <[email-masked]>
Date: Fri, 12 Jan 2007 10:54:54 +0100

Hi Juergen,

> > Windows that is different. It is the *nix implementation (at least
in
> my

I agree that it is a strong point, thus I mentioned it. I am just saying
we can not REQUIRE everyone to do the same on other platforms. And again
we are talking on things not happening on the wire. But I think that's
just a minor issue ;)

> For that, such a paper would be indeed useful.

I understand this and this is why I offered to write such a paper. But
the question remains if such a description belongs into a normative RFC.
Remember that the current discussion was spawned when David requested
that the architecture section in syslog-protocol is unclear and needs to
be extended. So far, my humble view is that *program architecture* does
NOT belong into a RFC. But this may be my inexperience and I am open to
good arguments why it should be...

Rainer

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Fri, 12 Jan 2007 11:36:33 +0100

> good arguments why it should be...

I do not think that Dave Harrington is talking about "program
architecture" or any implementation details.

A good example what Dave is looking for can be found in the Printer
MIB where they describe which componets make up a printer and how they
work together. Based on this model, they define the MIB module
itself. Such models, we sometimes like to call them information
models, really help to get management interfaces done right.

/js

---

From: "tom.petch" <[email-masked]>
Date: Sun, 14 Jan 2007 09:44:21 +0100

Mmm

I agree that management, SNMP or any kind, is only possible once we have agreed
a model - at some level - of what we are managing.  But I have no problems with
the model in RFC3164 that has been carried across into the current I-D  - except
one, which is for me the core of the problem.

So what is wrong with RFC3164?  I don't see it.  It does describe things I have
not seen in practice but have no problem with accepting that they exist and
should appear in the I-Ds.

I do find it telling that Rainer has a more limited definition of process than I
do ie I am happy to use it in the singular as a generic term for the various
bits of executing software in a something that emit a syslog message on the wire
whereas Rainer says no, that is several processes, we cannot refer to it as a
process singular.

Ditto device; RFC3164 ey seq. use it to describe something that emits a syslog
message on the wire and I will go along with that in the context of syslog.  So
when, David, you say you cannot map my terminology, of device (and syslogd) onto
that of the I-Ds, I am mystified.  I think I am using, and only using, the
terminogy of the I-Ds.

And if we cannot agree on this, then we cannot beign to specify how to manage
it, whatever it may be.

Tom Petch

----- Original Message -----
From: "David B Harrington" <[email-masked]>
To: "'Rainer Gerhards'" <[email-masked]>; <[email-masked]>
Sent: Friday, January 12, 2007 2:06 AM
Subject: RE: [Syslog] Doubts on definitions

> is referring From [email-masked] Sun Jan 14 06:11:03 2007
From: "tom.petch" <[email-masked]>
To: "David B Harrington" <[email-masked]>,
	<[email-masked]>
Subject: Re: [Syslog] Doubts on definitions
Date: Sun, 14 Jan 2007 09:44:21 +0100
Cc: 
Reply-To: "tom.petch" <[email-masked]>

Mmm

I agree that management, SNMP or any kind, is only possible once we have agreed
a model - at some level - of what we are managing.  But I have no problems with
the model in RFC3164 that has been carried across into the current I-D  - except
one, which is for me the core of the problem.

So what is wrong with RFC3164?  I don't see it.  It does describe things I have
not seen in practice but have no problem with accepting that they exist and
should appear in the I-Ds.

I do find it telling that Rainer has a more limited definition of process than I
do ie I am happy to use it in the singular as a generic term for the various
bits of executing software in a something that emit a syslog message on the wire
whereas Rainer says no, that is several processes, we cannot refer to it as a
process singular.

Ditto device; RFC3164 ey seq. use it to describe something that emits a syslog
message on the wire and I will go along with that in the context of syslog.  So
when, David, you say you cannot map my terminology, of device (and syslogd) onto
that of the I-Ds, I am mystified.  I think I am using, and only using, the
terminogy of the I-Ds.

And if we cannot agree on this, then we cannot beign to specify how to manage
it, whatever it may be.

Tom Petch

----- Original Message -----
From: "David B Harrington" <[email-masked]>
To: "'Rainer Gerhards'" <[email-masked]>; <[email-masked]>
Sent: Friday, January 12, 2007 2:06 AM
Subject: RE: [Syslog] Doubts on definitions

> > > What if the "application" is a limito. Maybe he is talking about multiple applications on a
> > > and architected-capability-device, such as
> an