LiS-2.18 performance ?

[e***@netscape.net]

Hello,

I'm curious about people impressions about LiS-2.18 performance.
Is it better comparing to LiS-2.16.18 ?

Are there any known 2.18 issues that can be fixed to improve performance?

My understanding is that in LiS-2.18 most(all?) of the queue processing
is done by LiS kernel threads and queuerun is never executed from
the driver tasklet context. That may result, I guess, in excessive process
switching overhead and poorer performance.
I might be missing something, though.

The other thing I noticed when I ran my tests on a 4 processor system
is that only one LiS thread accumulated CPU time:

root 9574 1 0 Dec01 ? 00:02:27 [LiS-2.18.0:0] <-------
root 9575 1 0 Dec01 ? 00:00:01 [LiS-2.18.0:1]
root 9576 1 0 Dec01 ? 00:00:00 [LiS-2.18.0:2]
root 9577 1 0 Dec01 ? 00:00:00 [LiS-2.18.0:3]

Is it the way it's supposed to be, or it's a bug?


I'd appreciate any comment/advices regarding performance issues on LiS-2.18.

--
Eugene




___________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com

[Dave Grothe]

LiS does not wake up additional queue run threads until there are
around a dozen queues to process in the list. It is not deemed to be
worth the task wakeup time for lesser amounts. I did some
performance testing and found that list lengths in that range were
about optimal.

You can't do put/service processing from interrupt level because of
having to "down" semaphores, which operation can only be performed
from process level.

I am sure that Brian's Fast STREAMS has a far superior solution to
these problems.

-- Dave

Post by e***@netscape.net
Hello,
I'm curious about people impressions about LiS-2.18 performance.
Is it better comparing to LiS-2.16.18 ?
Are there any known 2.18 issues that can be fixed to improve performance?
My understanding is that in LiS-2.18 most(all?) of the queue processing
is done by LiS kernel threads and queuerun is never executed from
the driver tasklet context. That may result, I guess, in excessive process
switching overhead and poorer performance.
I might be missing something, though.
The other thing I noticed when I ran my tests on a 4 processor system
root 9574 1 0 Dec01 ? 00:02:27 [LiS-2.18.0:0] <-------
root 9575 1 0 Dec01 ? 00:00:01 [LiS-2.18.0:1]
root 9576 1 0 Dec01 ? 00:00:00 [LiS-2.18.0:2]
root 9577 1 0 Dec01 ? 00:00:00 [LiS-2.18.0:3]
Is it the way it's supposed to be, or it's a bug?
I'd appreciate any comment/advices regarding performance issues on LiS-2.18.
--
Eugene
----------
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
<http://mail.netscape.com>http://mail.netscape.com

[e***@netscape.net]

Understand.
BTW, I appreciate LiS very much, as well as your many years of work supporting it.

--
Eugene

-----Original Message-----
From: Dave Grothe <***@gcom.com>
To: ***@netscape.net; linux-***@gsyc.escet.urjc.es
Sent: Fri, 02 Dec 2005 10:15:59 -0600
Subject: Re: [Linux-streams] LiS-2.18 performance ?


LiS does not wake up additional queue run threads until there are around a dozen queues to process in the list. It is not deemed to be worth the task wakeup time for lesser amounts. I did some performance testing and found that list lengths in that range were about optimal.

You can't do put/service processing from interrupt level because of having to "down" semaphores, which operation can only be performed from process level.

I am sure that Brian's Fast STREAMS has a far superior solution to these problems.

-- Dave

At 08:13 AM 12/2/2005, ***@netscape.net wrote:

Hello,

I'm curious about people impressions about LiS-2.18 performance.
Is it better comparing to LiS-2.16.18 ?

Are there any known 2.18 issues that can be fixed to improve performance?

My understanding is that in LiS-2.18 most(all?) of the queue processing
is done by LiS kernel threads and queuerun is never executed from
the driver tasklet context. That may result, I guess, in excessive process
switching overhead and poorer performance.
I might be missing something, though.

The other thing I noticed when I ran my tests on a 4 processor system
is that only one LiS thread accumulated CPU time:

root 9574 1 0 Dec01 ? 00:02:27 [LiS-2.18.0:0] <-------
root 9575 1 0 Dec01 ? 00:00:01 [LiS-2.18.0:1]
root 9576 1 0 Dec01 ? 00:00:00 [LiS-2.18.0:2]
root 9577 1 0 Dec01 ? 00:00:00 [LiS-2.18.0:3]

Is it the way it's supposed to be, or it's a bug?


I'd appreciate any comment/advices regarding performance issues on LiS-2.18.

--
Eugene






Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com
___________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com

[Brian F. G. Bidulock]

Eugene,

I ran pipe performance tests on 2.56GHz PIV 333MHz FSB, same box I ran
2.16.16 and 2.16.18 pipe performance test on last time (about 18 months
ago). I show 2.18 as comparable to 2.16.16. The performance increases
of 2.16.18 only showed improvements when message sizes were within a
FASTBUF.

Performance tests on RH7.2 2.4.20-28.7bigmem (SMP kernel I tested on
last time) shows LiS 2.18 STREAMS-based pipes clocking in at a dismal
13% when compared to Linux SVR3-style native pipes. 2.16.18 showed
about 20% 18 months ago, but only beneath 64-byte read/write sizes and
then fell back to about 10% after that.

Performances tests on Centos4 (RHEL4 clone) and FC4 show the performance
gains of the 2.6 kernel (and recent re-optimizing compilers) to be quite
significant.

On FC4 (a regparms kernel), the per-byte read/write latency drops from
about 750 ps (picoseconds) on RH7.2 and CL4 to about 500 ps on FC4. LiS
2.18 experiences a per-byte read/write latency drop from 1600 ps on
RH7.2 to 900 ps on CL4 and FC4. I attribute the gain on native to the
regparms FC4 kernel. I attribute the gain on LiS to the better
compilers (3.4.3 and 4.0) on CL4 and FC4 that better find their way
around cruft in the code.

Per message read/write delays for LiS 2.18 drops from 20 us on RH7.2 to
8 us on CL4 and FC4, while native pipes sit at around 2.5 us on all
three. I attribute the gain on LiS to the tighter scheduling latency
and O(1) scheduler of the 2.6 kernel. STREAMS-based pipes are far more
susceptible to scheduling latency.

Overall, when compared to native pipes, LiS 2.18 performs at 12.7% for
RH7.2, 28.1% for CL4 and a top end of 38.8% for FC4. The FC4 native
pipes really cruise, so 38.8% is quite good. I attribute the good FC4
results to the O(1) scheduler, the regparms kernel and the re-optimizing
GCC 4 compiler.

It is interesting that kernel improvements generate better performance
gains than could be accomplished within LiS with the changes from
2.16.16 to 2.16.18. There, it was only a 2x gain when compared to
native pipes and only beneath 64-byte writes. The FC4 improvesments are
across all messages sizes (tested linear with .999 correlation up to
4096 bytes).

So I suppose the story with LiS is, if you want the best performance use
a good 2.6 kernel. Because 2.16.18 only runs on a 2.6 kernel, 2.18 is
the better choice of the two for performance. If you are running on a
2.4 kernel; however, expect better performance from 2.16.18 at message
sizes within a FASTBUF.

Now, Linux Fast-STREAMS...

LfS (streams-0.7a.4) in the same performance tests relative to Linux
native pipes clocked in a 40%, 60% and 75% on RH7.2, CL4 and FC4 over
all message sizes. When compared to LiS at 13%, 28% and 39% in the
same tests, LfS performs 3.1x, 2.1x, 1.9x compared to LiS. The 3x
performance gain on 2.4 SMP over LiS 2.18 is quite impressive,
particularly when you consider that compared to native pipes, LfS runs
as fast on RH7.2 2.4 as LiS 2.18 runs on FC4. The other impressive
figure is that LfS on FC4 is running at 75% of the performance of a
native Linux pipe. This exceeds John Boyd's "impressed" threshold
(better than 50% native pipe performance).

LfS is the best performance choice on any kernel. Transitioning from
LiS 2.16.18 to LfS is a better performance choice than to LiS 2.18.
But then, that's why I called it "Fast".

I will send a separate note on some of my discoveries reagarding
performance on LiS and LfS.

Here is the raw (well, half-cooked) data: (obtained using the perftest
program included in the OpenSS7 LiS 2.18.2 release and the streams
0.7a.4 release):

Linear regression was performed on pipe throughput at 4, 8, 16, 32,
64, 128, 256, 512, 1024, 2048 and 4096 byte message sizes running the
pipe wide open (100% cpu utilization). Correlations were usually 99.9%
Slope is per-byte read/write delay, intercept is per message read/write
delay. The delay is y = mx + b, where x is the message size in bytes.

Per byte delay, slope, (picoseconds):

RH7.2 CL4 FC4
--------- -------- --------
LiS 1620 886 925
LfS 1230 919 987
Linux 760 750 482

Per write delay, intercept, (microseconds):

RH7.2 CL4 FC4
--------- -------- --------
LiS 19.20 7.72 7.83
LfS 6.54 3.57 4.02
Linux 2.43 2.17 3.03

--brian

Post by e***@netscape.net
Hello,
I'm curious about people impressions about LiS-2.18 performance.
Is it better comparing to LiS-2.16.18 ?
Are there any known 2.18 issues that can be fixed to improve
performance?
My understanding is that in LiS-2.18 most(all?) of the queue
processing
is done by LiS kernel threads and queuerun is never executed from
the driver tasklet context. That may result, I guess, in excessive
process
switching overhead and poorer performance.
I might be missing something, though.
The other thing I noticed when I ran my tests on a 4 processor system
root 9574 1 0 Dec01 ? 00:02:27 [LiS-2.18.0:0] <-------
root 9575 1 0 Dec01 ? 00:00:01 [LiS-2.18.0:1]
root 9576 1 0 Dec01 ? 00:00:00 [LiS-2.18.0:2]
root 9577 1 0 Dec01 ? 00:00:00 [LiS-2.18.0:3]
Is it the way it's supposed to be, or it's a bug?
I'd appreciate any comment/advices regarding performance issues on
LiS-2.18.
--
Eugene
_________________________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
[1]http://mail.netscape.com
References
1. http://mail.netscape.com/

[Brian F. G. Bidulock]

Eugene,

Here's the second note: some interesting things discovered during
testing.

Linux Fast-STREAMS performs at 85% of a pipe when the STREAMS scheduler
is run as a per-cpu software interrupt instead of a kernel thread, and
runs 3 times faster the LiS instead of just twice. The only problem is
that put and service procedures run at bottom half. That breaks the
strinet driver. So I set the default back to per-cpu kernel threads.
Theoretically there should only be a difference on non-preemptive
kernels (i.e. older 2.4 kernels).

LiS 2.18, like 2.16 before it, runs its kernel threads with FIFO
scheduling and a real-time priority of 50. Linux software interrupts
run nice -19 (as nice as they can get), but are executed upon return
from a hardware interrupt regardless of scheduling priority. I tried
running LfS kernel threads like LiS (SCHED_FIFO, priority 50) and it
SLOWED DOWN. This is too high a priority to run service procedures.
Try running them SCHED_RR nice -19 (but then other things might break
because races will change).

LiS 2.18, unlike 2.16 does not run the STREAMS scheduler on exit from
system calls. LfS runs the STREAMS scheduler in the current process
context on exit from system calls in accordance with SVR 4.2. This
removes a task switch for calls not subject to flow control in the
Stream and exhibits better performance (as well as running the put
procedure at the top of the module stack in user context which is
important to some non-conforming modules).

I also tested inter-module put and service procedure performance by
pushing a bunch of pass through or buffer modules onto the pipe. LfS
performance is only about 30% better than LiS. It appears that most of
LiS performance problems are in the Stream head and scheduling of
service procedures, rather than in put(), putnext() or service procedure
invocation itself.

It might be possible to get STREAMS-based pipes to outperform Linux
native pipes with LfS (but likely not with LiS). LfS uses memory
caches for all STREAMS structures, including the Stream head and queue
pairs. Linux native pipes kmalloc their pipe end data structures
(but page allocate their buffers). I think that if I run many pipes
in parallel, LfS could exhibit better performance than native pipes
because the Stream heads are memory cached whereas the native pipes
will cache miss on the pipe end structures. In a controlled test, LfS
can theoretically exhibit 150% of the per read/write performance of a
Linux native pipe. (Boy, would that impress John or what!) LfS could
exhibit 6x the performance of LiS under the same circumstances (LiS
performance enhancements don't include memory cache for Stream heads).

I haven't done SMP testing, primarily 'cause I don't have an SMP box:
anyone willing to donate an SMP box to the project can have LfS tested
on their platform of choice and get CVS archive access to boot. Failing
that, I am going to propose LfS to the OSDL for testing, but that will
probably take longer.

Unlike LiS, LfS runs a STREAMS scheduler per CPU differently: qenable
schedules on the same CPU for which it was invoked and STREAMS scheduler
threads run in their own per-CPU context (separate runqueues lists).
This also uses per-CPU thread info that greatly reduces lock contention
between CPUs. It would be quite interesting to perform multiple
parallel pipe comparison tests on an N-way box. I think LfS will really
shine there.

I will wrap the performance results into a proper report sometime,
but, I am furiously trying to wrap the documentation for LfS so that
it can be publicly released.

Hope that helps you in your quest for performance.

--brian

Post by e***@netscape.net
Hello,
I'm curious about people impressions about LiS-2.18 performance.
Is it better comparing to LiS-2.16.18 ?
Are there any known 2.18 issues that can be fixed to improve
performance?
My understanding is that in LiS-2.18 most(all?) of the queue
processing
is done by LiS kernel threads and queuerun is never executed from
the driver tasklet context. That may result, I guess, in excessive
process
switching overhead and poorer performance.
I might be missing something, though.
The other thing I noticed when I ran my tests on a 4 processor system
root 9574 1 0 Dec01 ? 00:02:27 [LiS-2.18.0:0] <-------
root 9575 1 0 Dec01 ? 00:00:01 [LiS-2.18.0:1]
root 9576 1 0 Dec01 ? 00:00:00 [LiS-2.18.0:2]
root 9577 1 0 Dec01 ? 00:00:00 [LiS-2.18.0:3]
Is it the way it's supposed to be, or it's a bug?
I'd appreciate any comment/advices regarding performance issues on
LiS-2.18.
--
Eugene
_________________________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
[1]http://mail.netscape.com
References
1. http://mail.netscape.com/

[e***@netscape.net]

Brian,

LiS 2.18, unlike 2.16 does not run the STREAMS scheduler on exit from system calls.

What I noticed in 2.18th lis_setqsched(can_call) function is that
the 'can_call' flag processing is completely gone from that function comparing with 2.16.18.

To measure impact of that 'can_call' thing I unconditionally set it to 0 in 2.16.18,
i.e. all queuerun() processing were done in LiS kernel threads, like in 2.18.0.
Performance of my loopback tests became slower by 25%.

Then in 2.18.0 I added the following couple of lines at the very beginning of lis_setqsched():

if ((can_call) && !(in_interrupt()))
{
lis_run_queues(my_cpu) ;
}

And performance gained 25% on my loopback test.

Q: Why can_call processing was removed and can we safely put it back?

thanks,
--
Eugene


___________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com

[Brian F. G. Bidulock]

eugenelisstreams,

SVR 4 calls runqueues() before a context switch (sleep or schedule) and
before returning to user mode (return from system call). When running a
kernel thread, the purpose is to avoid a context switch to the STREAMS
scheduler kernel thread.

LiS has a few problems in this regard that I alluded to in a previous
note: its kernel threads run at SCHED_FIFO with a priority of 50. Calling
runqueues() before exiting a system call will not avoid the context
switch (because the context switch will occur anyway: a real-time thread
has been woken). But your latency will be better, which I suppose explains
your 25%. If you combine that with dropping back to SCHED_RR and nice -19
on the kernel threads, then you can avoid the context switch altogether.

I suppose the only problem might be that some semaphore gets held while
queue are being run. Perhaps Dave could comment why it was removed in
the first place. (Removing it caused other problems, the put procedure
of the module at the top of the module stack used to execute in user
context in 2.16.18, but it executes in STREAMS scheduler context in 2.18.)

On 2.4 you will need over 350% more improvement to come even close to
Linux Fast-STREAMS.

I now have pipes at 47% on 2.4 SMP and 81% on FC4. I have one test
scenario (asynchronous read/write, size 4096 bytes) where LfS pipes run
at 110% of Linux native pipes. I show an inter-module performance on
FC4 for LfS that is twice that of LiS.

--brian

Post by e***@netscape.net
Brian,

LiS 2.18, unlike 2.16 does not run the STREAMS scheduler on exit

from system calls.
What I noticed in 2.18th lis_setqsched(can_call) function is that
the 'can_call' flag processing is completely gone from that function
comparing with 2.16.18.
To measure impact of that 'can_call' thing I unconditionally set it to 0 in 2.16.18,
i.e. all queuerun() processing were done in LiS kernel threads, like
in 2.18.0.
Performance of my loopback tests became slower by 25%.
Then in 2.18.0 I added the following couple of lines at the very
if ((can_call) && !(in_interrupt()))
{
lis_run_queues(my_cpu) ;
}
And performance gained 25% on my loopback test.
Q: Why can_call processing was removed and can we safely put it back?
thanks,
--
Eugene
_________________________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
[1]http://mail.netscape.com
References
1. http://mail.netscape.com/

[e***@netscape.net]

Post by Brian F. G. Bidulock
I suppose the only problem might be that some semaphore gets held while
queue are being run. Perhaps Dave could comment why it was removed in
the first place.

Dave, sorry to bother you, can you comment on why the 'can_call'
processing was removed from lis_setqsched() function in LiS-2.18 ?

I.e. in 2.16 lis_setqsched() could call lis_runqueues() directly if 'can_call'
parameter was set. In 2.18 everything is passed on to LiS threads.

Since performance is so much better with direct lis_runqueues() call
can I add it back? I.e. is it safe to add it back or there is something to consider
before doing this?

I'll really appreciate your comments.

Thanks,
--
Eugene

___________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com

[Dave Grothe]

I don't remember any more. I was probably concerned with multiple
executions of queue processing on the same CPU. One thread calls
lis_runqueues on the way out of a system call and then another thread
wakes up the queue runner thread for that CPU. Now you have two
instances running on the same CPU. Don't know if that causes problems or not.
-- Dave

Post by e***@netscape.net

Post by Brian F. G. Bidulock
I suppose the only problem might be that some semaphore gets held while
queue are being run. Perhaps Dave could comment why it was removed in
the first place.

Dave, sorry to bother you, can you comment on why the 'can_call'
processing was removed from lis_setqsched() function in LiS-2.18 ?
I.e. in 2.16 lis_setqsched() could call lis_runqueues() directly if 'can_call'
parameter was set. In 2.18 everything is passed on to LiS threads.
Since performance is so much better with direct lis_runqueues() call
can I add it back? I.e. is it safe to add it back or there is
something to consider
before doing this?
I'll really appreciate your comments.
Thanks,
--
Eugene
----------
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
<http://mail.netscape.com>http://mail.netscape.com

[Brian F. G. Bidulock]

eugenelisstreams,

A Linux Fast-STREAMS STREAMS-based pipe is now running at 106% of a
Linux native pipe on the latest FC4 2.6.14 kernel. LiS runs 19%.

--brian

Post by e***@netscape.net

I suppose the only problem might be that some semaphore gets held

while

queue are being run. Perhaps Dave could comment why it was removed

in

the first place.

Dave, sorry to bother you, can you comment on why the 'can_call'
processing was removed from lis_setqsched() function in LiS-2.18 ?
I.e. in 2.16 lis_setqsched() could call lis_runqueues() directly if
'can_call'
parameter was set. In 2.18 everything is passed on to LiS threads.
Since performance is so much better with direct lis_runqueues() call
can I add it back? I.e. is it safe to add it back or there is
something to consider
before doing this?
I'll really appreciate your comments.
Thanks,
--
Eugene
_________________________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
[1]http://mail.netscape.com
References
1. http://mail.netscape.com/

[e***@netscape.net]

Post by Brian F. G. Bidulock
A Linux Fast-STREAMS STREAMS-based pipe is now running at 106% of a
Linux native pipe on the latest FC4 2.6.14 kernel. LiS runs 19%.

Amazing! When do you plan to make it GA?

--
Eugene


___________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com

[Brian F. G. Bidulock]

eugenelisstreams,

Post by e***@netscape.net

Post by Brian F. G. Bidulock
A Linux Fast-STREAMS STREAMS-based pipe is now running at 106% of a
Linux native pipe on the latest FC4 2.6.14 kernel. LiS runs 19%.

Amazing! When do you plan to make it GA?

Well... I've said Monday for several weeks now. Now that the initial
performance and conformance testing is complete, I'm thinking... Monday. ;)

Of course, it is already available to subscribers and sponsors of the project.

--brian

Post by e***@netscape.net
--
Eugene
_________________________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
[1]http://mail.netscape.com
References
1. http://mail.netscape.com/

[Brian F. G. Bidulock]

eugenelisstreams,

Check out the performance report here:
http://www.openss7.org/streams_perf.html

You can get a copy of Linux Fast-STREAMS from the downloads page at
http://www.openss7.org/download.html

--brian

Post by e***@netscape.net

Post by Brian F. G. Bidulock
A Linux Fast-STREAMS STREAMS-based pipe is now running at 106% of a
Linux native pipe on the latest FC4 2.6.14 kernel. LiS runs 19%.

Amazing! When do you plan to make it GA?
--
Eugene
_________________________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
[1]http://mail.netscape.com
References
1. http://mail.netscape.com/

[e***@netscape.net]

Brian,

I'm a bit lost on that download page.

The Streams package I found is : streams-0.7a.4.tar.bz2.
It is designated as 'fully functional alpha release'.

Can you confirm that that is the latest FS release?

thanks
--
Eugene




-----Original Message-----
From: Brian F. G. Bidulock <***@openss7.org>
To: ***@netscape.net
Cc: linux-***@gsyc.escet.urjc.es
Sent: Mon, 23 Jan 2006 11:21:19 -0700
Subject: Re: [Linux-streams] LiS-2.18 performance ?


eugenelisstreams,

Check out the performance report here:
http://www.openss7.org/streams_perf.html

You can get a copy of Linux Fast-STREAMS from the downloads page at
http://www.openss7.org/download.html

--brian

Post by e***@netscape.net

Post by Brian F. G. Bidulock
A Linux Fast-STREAMS STREAMS-based pipe is now running at 106% of a
Linux native pipe on the latest FC4 2.6.14 kernel. LiS runs 19%.

Amazing! When do you plan to make it GA?
--
Eugene
_________________________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
[1]http://mail.netscape.com
References
1. http://mail.netscape.com/

--
Brian F. G. Bidulock Š The reasonable man adapts himself to the Š
***@openss7.org Š world; the unreasonable one persists in Š
http://www.openss7.org/ Š trying to adapt the world to himself. Š
Š Therefore all progress depends on the Š
Š unreasonable man. -- George Bernard Shaw Š
___________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com

[Brian F. G. Bidulock]

eugenelisstreams,

That's correct. You can find a little less busy listing here:

http://www.openss7.org/streams_pkg.html

--brian

Post by e***@netscape.net
Brian,
I'm a bit lost on that download page.
The Streams package I found is : streams-0.7a.4.tar.bz2.
It is designated as 'fully functional alpha release'.
Can you confirm that that is the latest FS release?
thanks
--
Eugene

[e***@netscape.net]

Brian,

Does it work on 64-bit RH?

What kind of licensing is attached to LFS? Any support?

--
Eugene

-----Original Message-----
From: Brian F. G. Bidulock <***@openss7.org>
To: ***@netscape.net
Cc: linux-***@gsyc.escet.urjc.es
Sent: Tue, 31 Jan 2006 09:58:40 -0700
Subject: Re: [Linux-streams] LiS-2.18 performance ?


eugenelisstreams,

That's correct. You can find a little less busy listing here:

http://www.openss7.org/streams_pkg.html

--brian

Post by e***@netscape.net
Brian,
I'm a bit lost on that download page.
The Streams package I found is : streams-0.7a.4.tar.bz2.
It is designated as 'fully functional alpha release'.
Can you confirm that that is the latest FS release?
thanks
--
Eugene

--
Brian F. G. Bidulock Š The reasonable man adapts himself to the Š
***@openss7.org Š world; the unreasonable one persists in Š
http://www.openss7.org/ Š trying to adapt the world to himself. Š
Š Therefore all progress depends on the Š
Š unreasonable man. -- George Bernard Shaw Š
___________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com

[Brian F. G. Bidulock]

eugenelisstreams,

I haven't tested on 64-bit yet. (No machine.) In fact I have
only tested on ia32 and 32-bit ppc UP. Different architectures
and N-way machines was one of the things that I was hoping the
OSDL would do. Anyone willing to donate a machine for testing,
or to donate the time to simply run the test suites on report any
problems, would be appreciated.

LFS is licensed under GPL. IANAL, but the GPL does not impact
your code unless it can be construed to be derived from LFS.
Also, commercial licensing is also available from OpenSS7
Corporation, if needed. mailto:***@openss7.com

I am maintaining and developing Linux Fast-STREAMS. All of the
OpenSS7 Project software has been converted to work with Linux
Fast-STREAMS and at some point in the near future will no longer
be supported on LiS, (during comparison testing, LiS was locking
the processor so often as to be really annoying). Problems and
patches can be reported to me, here, or on the OpenSS7 mailing
lists at http://www.openss7.org/mailinglist.html

If you prefer commercial support, it can be contracted from
OpenSS7 Corporation. mailto:***@openss7.com

--brian

Post by e***@netscape.net
Brian,
Does it work on 64-bit RH?
What kind of licensing is attached to LFS? Any support?
--
Eugene
-----Original Message-----
Sent: Tue, 31 Jan 2006 09:58:40 -0700
Subject: Re: [Linux-streams] LiS-2.18 performance ?
eugenelisstreams,
[1]http://www.openss7.org/streams_pkg.html
--brian

[e***@netscape.net]

Ok. Got it.

I'll do my best to try it on 64-bit RH 2.6.9-5.ELsmp.

--
Eugene


-----Original Message-----
From: Brian F. G. Bidulock <***@openss7.org>
To: ***@netscape.net
Cc: linux-***@gsyc.escet.urjc.es
Sent: Tue, 31 Jan 2006 11:15:03 -0700
Subject: Re: [Linux-streams] LiS-2.18 performance ?


eugenelisstreams,

I haven't tested on 64-bit yet. (No machine.) In fact I have
only tested on ia32 and 32-bit ppc UP. Different architectures
and N-way machines was one of the things that I was hoping the
OSDL would do. Anyone willing to donate a machine for testing,
or to donate the time to simply run the test suites on report any
problems, would be appreciated.

LFS is licensed under GPL. IANAL, but the GPL does not impact
your code unless it can be construed to be derived from LFS.
Also, commercial licensing is also available from OpenSS7
Corporation, if needed. mailto:***@openss7.com

I am maintaining and developing Linux Fast-STREAMS. All of the
OpenSS7 Project software has been converted to work with Linux
Fast-STREAMS and at some point in the near future will no longer
be supported on LiS, (during comparison testing, LiS was locking
the processor so often as to be really annoying). Problems and
patches can be reported to me, here, or on the OpenSS7 mailing
lists at http://www.openss7.org/mailinglist.html

If you prefer commercial support, it can be contracted from
OpenSS7 Corporation. mailto:***@openss7.com

--brian

Post by e***@netscape.net
Brian,
Does it work on 64-bit RH?
What kind of licensing is attached to LFS? Any support?
--
Eugene
-----Original Message-----
Sent: Tue, 31 Jan 2006 09:58:40 -0700
Subject: Re: [Linux-streams] LiS-2.18 performance ?
eugenelisstreams,
[1]http://www.openss7.org/streams_pkg.html
--brian

--
Brian F. G. Bidulock Š The reasonable man adapts himself to the Š
***@openss7.org Š world; the unreasonable one persists in Š
http://www.openss7.org/ Š trying to adapt the world to himself. Š
Š Therefore all progress depends on the Š
Š unreasonable man. -- George Bernard Shaw Š
___________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com

[e***@netscape.net]

Brian,

Post by Brian F. G. Bidulock
I haven't tested on 64-bit yet.

I'm not familiar with LFS download, unpacking, installing procedures.

Could you provide brief instructions on how to do this?

Do I have to do anything special to install and compile it.

BTW, what are you expectations with regard to 64-bit issues?
Should it work or some tuning will be needed.

thanks,
--
Eugene


-----Original Message-----
From: Brian F. G. Bidulock <***@openss7.org>
To: ***@netscape.net
Cc: linux-***@gsyc.escet.urjc.es
Sent: Tue, 31 Jan 2006 11:15:03 -0700
Subject: Re: [Linux-streams] LiS-2.18 performance ?


eugenelisstreams,

I haven't tested on 64-bit yet. (No machine.) In fact I have
only tested on ia32 and 32-bit ppc UP. Different architectures
and N-way machines was one of the things that I was hoping the
OSDL would do. Anyone willing to donate a machine for testing,
or to donate the time to simply run the test suites on report any
problems, would be appreciated.

LFS is licensed under GPL. IANAL, but the GPL does not impact
your code unless it can be construed to be derived from LFS.
Also, commercial licensing is also available from OpenSS7
Corporation, if needed. mailto:***@openss7.com

I am maintaining and developing Linux Fast-STREAMS. All of the
OpenSS7 Project software has been converted to work with Linux
Fast-STREAMS and at some point in the near future will no longer
be supported on LiS, (during comparison testing, LiS was locking
the processor so often as to be really annoying). Problems and
patches can be reported to me, here, or on the OpenSS7 mailing
lists at http://www.openss7.org/mailinglist.html

If you prefer commercial support, it can be contracted from
OpenSS7 Corporation. mailto:***@openss7.com

--brian

Post by Brian F. G. Bidulock
Brian,
Does it work on 64-bit RH?
What kind of licensing is attached to LFS? Any support?
--
Eugene
-----Original Message-----
Sent: Tue, 31 Jan 2006 09:58:40 -0700
Subject: Re: [Linux-streams] LiS-2.18 performance ?
eugenelisstreams,
[1]http://www.openss7.org/streams_pkg.html
--brian

--
Brian F. G. Bidulock Š The reasonable man adapts himself to the Š
***@openss7.org Š world; the unreasonable one persists in Š
http://www.openss7.org/ Š trying to adapt the world to himself. Š
Š Therefore all progress depends on the Š
Š unreasonable man. -- George Bernard Shaw Š
___________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com

[Brian F. G. Bidulock]

Eugene,

Sorry for the delay, I was in transit...

Post by e***@netscape.net
Brian,

Post by Brian F. G. Bidulock
I haven't tested on 64-bit yet.

I'm not familiar with LFS download, unpacking, installing procedures.
Could you provide brief instructions on how to do this?

See

http://www.openss7.org/STREAMS_manual.html#Installation

This manual is also contained in the distribution in (doc/manual/)
STREAMS.pdf and STREAMS.html.

Post by e***@netscape.net
Do I have to do anything special to install and compile it.

Not much. The nitty gritty stuff is in the manual above, but it is
simple enough to do. All OpenSS7 releases are now packaged with
GNU/autoconf so the usual ./configure, make, make install will do.

So, basically:

wget http://www.openss7.org/tarballs/streams-0.7a.4.tar.bz2
tar -xjvf streams-0.7a.4.tar.bz2
mkdir build
cd build
../streams-0.7a.4/configure
make compile.log
sudo make install.log
sudo make installcheck

Cross-builds, particularly for embedded targets is a little more
complicated. For that, see

http://www.openss7.org/STREAMS_manual.html#Building-from-the-Tar-Ball

For instructions on running the pre-installation checks and the
post-installation test suites, see:

http://www.openss7.org/STREAMS_manual.html#Troubleshooting

Post by e***@netscape.net
BTW, what are you expectations with regard to 64-bit issues?

64 on 64 should be ok.

LFS does provide proper alignment on primary STREAMS data structures
(such as adjusting the FASTBUF size from 64 bytes to 128 bytes on 64 bit
and overlap in iocblk, copyreq and copyresp structures).

Post by e***@netscape.net
Should it work or some tuning will be needed.

Problems will ensue on 32 over 64 or 64 over 32 because USL header files
use long and ulong: some _LP64_ checks to make these uint32_t and
int32_t are needed. Also LFS does not yet register 32 to 64 or 64 to 32
bit ioctl conversion functions, nor does it build 32 over 64 or 64 over
32 libraries.

I'm hoping to get a Dual-Xeon and/or Dual-Athlon server here for
testing. I might be able to get a 64-bit ppc system here if I coax a
sponsor. But you have 64-bit ppc with multiple processors?

--brian

[e***@netscape.net]

Thanks, I'll give it a try, probably next week.

Post by Brian F. G. Bidulock
But you have 64-bit ppc with multiple processors?

What I have is: MP Dell box with 'Kernel 2.6.9-5.ELsmp on an x86_64'.

--
Eugene



___________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com

[Brian F. G. Bidulock]

Eugene,

Are those dual Xeon? 7525AF2 or something?

--brian

Post by e***@netscape.net
Thanks, I'll give it a try, probably next week.

Post by Brian F. G. Bidulock
But you have 64-bit ppc with multiple processors?

What I have is: MP Dell box with 'Kernel 2.6.9-5.ELsmp on an x86_64'.
--
Eugene
_________________________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
[1]http://mail.netscape.com
References
1. http://mail.netscape.com/

[e***@netscape.net]

Post by Brian F. G. Bidulock
Are those dual Xeon? 7525AF2 or something?

It is:
Dell Precision 670n Mini-Tower (Dual)3.20GHz XEON Processor
w/ 2MB L2 Cache, 800FSB (221-7985)

--
Eugene

___________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com

[Brian F. G. Bidulock]

Eugene,

I've been trying to get a 4-way dual Xeon 3.0GHz system on the
7520 chipset here for testing, and maybe a 2-way EMT64 630 HT
system as well. I noticed that dual Opteron systems appeart to
be becoming popular.

Are most people still working with dual Xeon Intel server boards
or is there interest in the Opteron systems as well?

As always, hardware contributions are welcome.

--brian

Post by e***@netscape.net

Post by Brian F. G. Bidulock
Are those dual Xeon? 7525AF2 or something?

Dell Precision 670n Mini-Tower (Dual)3.20GHz XEON Processor
w/ 2MB L2 Cache, 800FSB (221-7985)
--
Eugene
_________________________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
[1]http://mail.netscape.com
References
1. http://mail.netscape.com/

[e***@netscape.net]

Brian, I did not have a chance to test LFS this week.
Next week probably.

I'll keep you posted.

-
Eugene

___________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com

[e***@netscape.net]

Post by Brian F. G. Bidulock
wget http://www.openss7.org/tarballs/streams-0.7a.4.tar.bz2
tar -xjvf streams-0.7a.4.tar.bz2
mkdir build
cd build
../streams-0.7a.4/configure
make compile.log
sudo make install.log
sudo make installcheck

I stopped before 'sudo make install.log' step and observed streams.ko and bunch
of other .ko modules in the build directory.
Before installing all of that can I just do 'insmod streams.ko' to see how is it going?

Also, after install how do I remove all of that?

--
Eugene

___________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
http://mail.netscape.com

[Brian F. G. Bidulock]

Eugene,

'make check' before 'make install' will perform some
symbol checks, but I have already run against that
kernel in CentOS4 and WBEL4 clones of RHEL4.

'sudo make uninstall.log' uninstalls and tees output to
uninstall.log. The install and uninstall processes are
autoconf based and are clean and good at reversing the
installation process.

One warning: be careful about installing on a machine
that has any release of LiS installed (because LiS will
be largely removed during the installation process due to
conflicts).

So you can do (assuming you have sudo, su to root otherwise):

wget http://www.openss7.org/tarballs/streams-0.7a.4.tar.bz2
tar -xjvf streams-0.7a.4.tar.bz2
mkdir build
cd build
../streams-0.7a.4/configure
make compile.log
make check.log # <-- pre-installation checks
sudo make install.log # <-- installs everything
sudo make installcheck # <-- post-installation test suite
sudo make retest # <-- if you have some sporadic test case failure
sudo make uninstall.log # <-- removes it all
cd ..
rm -fr build # <-- remove build directory
rm -fr streams-0.7a.4 # <-- remove source directory
rm streams-0.7a.4.tar.bz2 # <-- remove tarball

That should take you from scratch to scratch as though it was
never installed on the system.

If you want closer control, you can build rpms from the srpm as
described in the manual and then use rpm -ivvv and rpm -evvv to
ensure that the system is returned to its previous state. But
it is harder to run the standalone testsuites (installed in
/usr/libexec/streams).

--brian

wget [1]http://www.openss7.org/tarballs/streams-0.7a.4.tar.bz2
tar -xjvf streams-0.7a.4.tar.bz2
mkdir build
cd build
../streams-0.7a.4/configure
make compile.log
sudo make install.log
sudo make installcheck

I stopped before 'sudo make install.log' step and observed streams.ko
and bunch
of other .ko modules in the build directory.
Before installing all of that can I just do 'insmod streams.ko' to
see how is it going?
Also, after install how do I remove all of that?
--
Eugene
_________________________________________________________________
Try the New Netscape Mail Today!
Virtually Spam-Free | More Storage | Import Your Contact List
[2]http://mail.netscape.com
References
1. http://www.openss7.org/tarballs/streams-0.7a.4.tar.bz2
2. http://mail.netscape.com/

[Rodrigues, Edward]

Hi Brian,

I was looking into LFS and just wondering whether you have created any
migration document from LiS to LFS.
LiS was converting .o to .ko, instead of using .ko directly. Looks like
LFS uses .ko

Thanks
Ed



-----Original Message-----
From: linux-streams-***@gsyc.escet.urjc.es
[mailto:linux-streams-***@gsyc.escet.urjc.es] On Behalf Of Brian F. G.
Bidulock
Sent: Tuesday, January 31, 2006 1:15 PM
To: ***@netscape.net
Cc: linux-***@gsyc.escet.urjc.es
Subject: Re: [Linux-streams] LiS-2.18 performance ?

eugenelisstreams,

I haven't tested on 64-bit yet. (No machine.) In fact I have
only tested on ia32 and 32-bit ppc UP. Different architectures
and N-way machines was one of the things that I was hoping the
OSDL would do. Anyone willing to donate a machine for testing,
or to donate the time to simply run the test suites on report any
problems, would be appreciated.

LFS is licensed under GPL. IANAL, but the GPL does not impact
your code unless it can be construed to be derived from LFS.
Also, commercial licensing is also available from OpenSS7
Corporation, if needed. mailto:***@openss7.com

I am maintaining and developing Linux Fast-STREAMS. All of the
OpenSS7 Project software has been converted to work with Linux
Fast-STREAMS and at some point in the near future will no longer
be supported on LiS, (during comparison testing, LiS was locking
the processor so often as to be really annoying). Problems and
patches can be reported to me, here, or on the OpenSS7 mailing
lists at http://www.openss7.org/mailinglist.html

If you prefer commercial support, it can be contracted from
OpenSS7 Corporation. mailto:***@openss7.com

--brian

Post by e***@netscape.net
Brian,
Does it work on 64-bit RH?
What kind of licensing is attached to LFS? Any support?
--
Eugene
-----Original Message-----
Sent: Tue, 31 Jan 2006 09:58:40 -0700
Subject: Re: [Linux-streams] LiS-2.18 performance ?
eugenelisstreams,
[1]http://www.openss7.org/streams_pkg.html
--brian

[Brian F. G. Bidulock]

Ed,

The build process is identical to the OpenSS7 LiS 2.18.2 release
and follows the modpost process on 2.6 kernels. That is,
regular .o files are compiled 'gcc -c' and then export sections
are created using the 'scripts/modpost' sh script, the resulting
sections are compiled and combined with load script into the
final .ko object.

The build uses a modpost sh script instead of a the C program
present in the kernel to permit cacheing of symbol values as
well as to avoid host compiling when cross-compiling (autoconf
doesn't like to do that very much). Also, it is the only way to
support pre-2.6.10 and post-2.6.10 approaches in the same
object.

The strcompat package provides a precise source interface and a
close binary interface for running LiS (2.18.0 or 2.18.2)
modules on Linux Fast-STREAMS. It also provides a source
interface for AIX, HPUX, IRIX, OSF/1, MacOT, Solaris, SUX,
SVR3.2, SVR4, UnixWare, UXP, others. I'm sorry but there is no
binary packaging utility in either LiS 2.18.2 or streams 0.7a.4
(the OpenSS7 projects releases as source and has no need for
it).

The 'scripts/strconf-sh' script that reads LiS and LfS
configuration files, accepts a --pkgobject flag that can be used
to generate a stub .c file that can generate a stub .o file that
can be linked with a pre-compiled binary to create a loadable
kernel module, but the process is not yet automated.

If we receive sponsorship at an adequate level I would entertain
writing a 'strbin' package that would generally wrap binary .o
files into kernel modules with "Proprietary" licensing, similar
to what has been done in the past with LiS. Otherwise, anyone
wishing to release partial binaries will have to provide their
own final linking process.

I'm sorry but there is no migration document. Take a look at
the strinet package for an example of how to modify a driver to
work with both LiS and LfS.

--brian

Post by Rodrigues, Edward
Hi Brian,
I was looking into LFS and just wondering whether you have created any
migration document from LiS to LFS.
LiS was converting .o to .ko, instead of using .ko directly. Looks like
LFS uses .ko
Thanks
Ed
-----Original Message-----
Bidulock
Sent: Tuesday, January 31, 2006 1:15 PM
Subject: Re: [Linux-streams] LiS-2.18 performance ?
eugenelisstreams,
I haven't tested on 64-bit yet. (No machine.) In fact I have
only tested on ia32 and 32-bit ppc UP. Different architectures
and N-way machines was one of the things that I was hoping the
OSDL would do. Anyone willing to donate a machine for testing,
or to donate the time to simply run the test suites on report any
problems, would be appreciated.
LFS is licensed under GPL. IANAL, but the GPL does not impact
your code unless it can be construed to be derived from LFS.
Also, commercial licensing is also available from OpenSS7
I am maintaining and developing Linux Fast-STREAMS. All of the
OpenSS7 Project software has been converted to work with Linux
Fast-STREAMS and at some point in the near future will no longer
be supported on LiS, (during comparison testing, LiS was locking
the processor so often as to be really annoying). Problems and
patches can be reported to me, here, or on the OpenSS7 mailing
lists at http://www.openss7.org/mailinglist.html
If you prefer commercial support, it can be contracted from
--brian

Post by e***@netscape.net
Brian,
Does it work on 64-bit RH?
What kind of licensing is attached to LFS? Any support?
--
Eugene
-----Original Message-----
Sent: Tue, 31 Jan 2006 09:58:40 -0700
Subject: Re: [Linux-streams] LiS-2.18 performance ?
eugenelisstreams,
[1]http://www.openss7.org/streams_pkg.html
--brian

--
Brian F. G. Bidulock | The reasonable man adapts himself to the |
http://www.openss7.org/ | trying to adapt the world to himself. |
| Therefore all progress depends on the |
| unreasonable man. -- George Bernard Shaw |
_______________________________________________
Linux-streams mailing list
http://gsyc.escet.urjc.es/mailman/listinfo/linux-streams
_______________________________________________
Linux-streams mailing list
http://gsyc.escet.urjc.es/mailman/listinfo/linux-streams