NeWS – Network Extensible Window System
stevewilhelm
9 years ago
86
74
https://en.wikipedia.org/wiki/NeWS
tilt_error9 years ago
PostScript really is a lovely programming language. Some 20 years ago, I used a development environment (PSAlter) that made it quite palatable -- visualising dictionaries and stacks -- and made it possible to step through programs.

There exists a nice free e-book describing PostScript (Thinking in PostScript by Glenn C. Reid) [1].

[1] https://web.archive.org/web/20000914161447/http://www.rightbrain.com/download/books/ThinkingInPostScript.pdf

EDIT

While looking for the Thinking in PostScript book mentioned above, I stumbled upon this legal online archive for books on the net [2] which may be of interest to others.

[2] http://onlinebooks.library.upenn.edu

tonyedgecombetilt_error9 years ago
DonHopkinstilt_error9 years ago
That's very cool! I'd like to see more about how the debugger works. Are there any video demos available, please?

I love Glenn Reid's book, Thinking PostScript. It's like the Thinking Forth [1] of PostScript! ;) I always thought of PostScript as a cross between Lisp and Forth [2], but more in the Lisp camp, since it's so easy to write a metacircular evaluator [3] or a quine [4] in PostScript:

    {{[ exch /dup load /exec load ] cvx} dup exec}
Do you remember Glenn Reid's "Distillery" [5]? It's a PostScript program that optimized other PostScript programs, which you prepended before your PostScript print file, sent to a printer, and it partially evaluated the PostScript code in the print file. It then echoed back another highly optimized flat PostScript program that printed the exact same pages, all in one coordinate system, with superfluous graphics state changes removed. That glorious hack, based on John Warnock's idea of redefining the imaging operators, eventually evolved into PDF! [6]

Glenn also wrote a delightful "Font Appreciation" app for NeXT called TouchType [7], which decades later has only recently found its way into Photoshop.

I wrote a visual interactive PostScript debugging tool for NeWS called the PSIBER Space Deck [8], and the PseudoScientific Visualizer [9] for drawing PostScript data structures like NeWS object and class dictionaries or a network map of ARPANET IMPs, and a paper [10] about them. It was most useful for debugging itself, which was fortunate, since it was pretty buggy. I integrated the metacircular PostScript evaluator with the debugger, so you could single step through PostScript code, and draw graphics to visualize the program execution state. And I made my own NeWS distillery [11] "in the spirit of Glenn Reid's Distillery" ;), for making flat printable PostScript dumps of anything procedurally drawn on the screen, which I used for some of the illustrations in the paper [12].

[1] Thinking Forth: http://thinking-forth.sourceforge.net/

[2] PostScript -vs- Forth: http://donhopkins.com/home/code/ps-vs-forth.txt

[3] PostScript Metacircular Evaluator: http://donhopkins.com/home/code/ps.ps.txt

[4] PostScript Quine: http://donhopkins.com/home/archive/news-tape/fun/quine/quine

[5] Glenn Reid's PostScript Distillery: http://donhopkins.com/home/archive/postscript/newerstill.ps.txt

[6] The Distiller Story: http://blogs.adobe.com/insidepdf/2007/11/the_distiller_story.html

[7] TouchType: https://ftp.nice.ch/peanuts/GeneralData/Usenet/news/1990/_CSN-90/comp-sys-next/1990/Jul/_BaNG-%234-meeting-review.html

[8] PSIBER Space Deck Demo: https://www.youtube.com/watch?v=iuC_DDgQmsM

[9] Pseudo Scientific Visualizer Demo: https://www.youtube.com/watch?v=_fqCeuue5Ac

[10] The Shape of PSIBER Space: PostScript Interactive Bug Eradication Routines: http://www.donhopkins.com/drupal/node/97

[11] NeWS Distillery: http://donhopkins.com/home/archive/news-tape/utilities/cyber/distill.ps.txt

[12] Distilled PostScript PseudoScientific Visualization of Arpanet IMPS: http://donhopkins.com/home/archive/news-tape/documents/psiber/10.ps.txt

zxv9 years ago
Speaking of interesting mid 80s window systems, there was a light weight window system developed by Stephen Uhler at Belcore.

It is under 20K lines of C, and very easy to port. It is able to run over a serial line or a socket, in a few hundred K of RAM.

Screenshot: http://www.hack.org/mc/mgr/images/mgrscreen.png

More info: http://www.hack.org/mc/mgr/

"MGR is to X as Unix is to Multix." -Dan Nachbar

gattilorenzzxv9 years ago
I have recently ran it on Xenix, on a 386 system. Quite impressive, indeed.
cmrdporcupinezxv9 years ago
This was ported to the Atari ST/TT in the early 90s, and I ran it there. And then for a while it was the only practical option on my early (pre 1.0) Linux because X11 drivers for my Trident video card were not done.

Some neat ideas. Nice and minimalist.

cm3zxv9 years ago
MGR never supported X clients, right?
kjs3cm39 years ago
Correct...totally different way of looking at the problem.
rwmjcm39 years ago
No. The cool thing about MGR was that you programmed it by sending terminal-like control codes to stdout. (I used to run MGR on Minix).
arethuzarwmj9 years ago
Neat - even works that way for input as well:

"MGR notifies a client program of an event by sending it an ASCII character string in a format specified by the client program."

kjs3zxv9 years ago
MGR was very good and very Unix-y, multiplexing windows over ptys and presenting a pretty standard terminal abstraction to the rest of the system. It also ran on OS/9 and some other odd things. It's very lightweight; a nice options for embedded systems.
cer3149 years ago
Just this morning I spoke of NeWS (for the first time in at least 20 years) to a colleague at work. Very strange to see this posting on HN in the same day... PostScript with classes... and input methods... seemed to be a very interesting way to model user-interfaces, and graphics rendering in general...
tracker19 years ago
This just reminds me how much of a shame it is that the best option for a remote UI today is a localized browser, and the tools that go with it... Don't get me wrong, I think the browser/web is a great platform, and better than many other options that have come and gone, that said...

It sucks that VNC, X11, and RDP are the best we have... though, I will admit RDP is pretty serviceable, X11 just seems too janky to work over a really remote connection... and VNC, well it works, but, sigh...

I don't think that any remote application UI will really take over as everyone that might work on such a thing wants to gain a lock in with the goal of financial gain... so nothing really reaches a significant market share. This is the case were something that is great and libre/opensource would be better than a standard.

twstedtracker19 years ago
> X11 just seems too janky to work over a really remote connection.

Surely It is.

But I cannot forget how magic it seemed in the 80s doing

  titan$ export DISPLAY=zeus:0.0; xeyes &
over slow serial lines.
arethuzatwsted9 years ago
I cringe to think that I used to use xhost to allow access from any client - don't think there were many firewalls on JANET so anybody could have been connecting to the Sun (later DEC Alpha) I used.
pantulistwsted9 years ago
And the next thing would be something like

zeus$ xhost +

;)

gpvostwsted9 years ago
xmeltdown
DonHopkinsgpvos9 years ago
Jeremy Huxtable made the original "Big Brother" NeWS version of eyes [1], which inspired xeyes at SIGGRAPH '88 [2], and he also made the NeWS version of melt [3], in just 30 lines of code!

My favorite OpenWindows demo was Pat Lashley's monaeyes [4], of course. And I made a fancy version of eyeball windows for The NeWS Toolkit [5] to show off its drag-n-drop and nested window management capabilities, where the eyeball was the window frame, and the iris was the client window, and you could split them in two, and even drag and drop eyes inside of each other, so that sub-eye window frame was reparented inside the moving super-eye client iris! The effect was very creepy and CPU intensive, and I could never quite talk the OPEN LOOK folks into adopting them into the standard.

[1] http://donhopkins.com/home/archive/news-tape/fun/eye/eye.ps.txt

[2] http://www.x.org/archive/X11R7.5/doc/man/man1/xeyes.1.html

[3] http://donhopkins.com/home/archive/news-tape/fun/melt/melt.ps.txt

[4] http://donhopkins.com/home/archive/NeWS/monaeyes.txt

[5] http://donhopkins.com/home/archive/NeWS/eyes.ps.txt

kjs3twsted9 years ago
I ran X over a 128Kbps ISDN line for years. I thought it was pretty usable, as long as you were a bit careful about what you ran over it. XTerms were fine; things like Framemaker were painful.
tracker1twsted9 years ago
Agreed, and in the days of simpler apps, and 800x600/1024x768 displays being more usual, not as bad. I really do miss a lot of this.

I mean, the web is great for a lot of stuff (as I said)... but I feel it is a shame that we don't have a modern alternative for client/server applications... and every time I see one that has a lot of potential, it usually has some strings that will prevent adoption.

creshaltracker19 years ago
RDP is the best we have… and at the same time one of the worst kitchen sink protocols I've seen.

• Encryption can be completely disabled, forcing plaintext login – to comply with French government orders at some point in the 90s. It's still implemented in a lot of clients, because why not?

• Serial and parallel port forwarding (suddenly, USB over JS doesn't seem that surprising any more…)

• SMB share forwarding (complete with 8-byte-name-limit)

• SMB printer forwarding (as opposed to parallel port printer forwarding). rdesktop's manpage helpfully states that "Keep in mind that you need a 100% match in the server environment, or the driver will fail." Charming!

PCI device forwarding. Holy mother of DMA attacks!

You know, WebRTC doesn't seem too bad…

pjc50creshal9 years ago
PCI device forwarding. Holy mother of DMA attacks!

Could you give me a citation for this please? Google is unhelpful and only refers to "PCI compliance" which is not at all the same thing (and also a problem for different reasons).

Zardoz84tracker19 years ago
The last time that I used XN (X11 + deltas + intelligent cache + other extras) worked pretty well on remote. Even works fine over ADSL of 1MB .
fulafeltracker19 years ago
There's the question of cause and effect in the gradual degradation of the remote X11 experience. A lot of it depends on how the X client is written and pays attention to protocol round trips. In the old days people complained and filed bugs on apps if there was a slowdown in remote X11 performance because many people routinely ran heavy X11 apps on beefy servers and displayed on a wimpy machine.

RDP and VNC are immune to this because they are just glorified animated gif streams, but NeWS would be prone to the same problem if it was alive today.

digi_owlfulafel9 years ago
Shit hit the X11 fan as the DEs adopted OpenGL as an UI accelerator, effectively bypassing X for most tasks.

CADT at work yet again it seems, as it the code could iterate faster if they were not bound by having the OGL traffic go through the X infrastructure.

From there is has been a hop skip to Wayland, where X is seen as a slowly withering appendage.

datenwolfdigi_owl9 years ago
> Shit hit the X11 fan as the DEs adopted OpenGL as an UI accelerator, effectively bypassing X for most tasks.

That's not correct. If DEs actually were using OpenGL for UI acceleration, then at least for GL-2.1 and earlier everything would be sent as pure drawing commands to the display server by means of the X11/GLX transport protocol (indirect rendering context).

What actually happened was, that for some reason the developers of the toolkits thought, that it was somehow necessary to render everything in the client into a pixmap and only as a last step pass this through the X11 to the graphics hardware, thereby degrading the X11 server to a glorified bit-blitting machine.

That's why the design of Wayland is how it is, because toolkits developers couldn't be bothered to actually using the X11 drawing primitives in a clever way. And if you're not a total moron you can make it look extremely good. One thing that I never understood is, why antialiasing was never enabled for X core drawing primitives; it's not like you couldn't use raster ops with antialiased geometry either (like the XOR overdraw method).

digi_owldatenwolf9 years ago
That was basically what i was talking about in the second part. They started out with GLX, but then switched to KMS and going directly to the hardware. Best i can tell, this was done because GLX was seen as holding them back somehow.

Then again i have found the whole compositor thing unreliable at best. I still have it turned off on my (aging) XFCE install.

Anyways, i think the primary reason they want X11 dead is the whole security thing.

It seems there is not a week going by without some blog or other warning about how anything can sniff and/or insert keystrokes in X11, never mind screen scraping, and how Wayland will fix all that.

Never mind that one man's security issue is another man's feature...

datenwolfdigi_owl9 years ago
> but then switched to KMS and going directly to the hardware

That's not how it works. KMS is something display servers (X11, Wayland compositor) use, not something a client application should or would use (too impractical.

digi_owldatenwolf9 years ago
Yeah i guess i got that mixed up with DRI, though i can't say i payed much attention to either until all the hoopla about Wayland started.
datenwolfdigi_owl9 years ago
> Yeah i guess i got that mixed up with DRI, though.

DRI only comes into play if you're creating a direct OpenGL context running one of the open source OpenGL drivers (Mesa + DRM).

It does nothing for regular X11 clients, that don't use OpenGL. Actually DRI does not do drawing at all, it's mostly just a way for bypassing the X11 server for direct OpenGL contexts as specified in the GLX specification. Essentially it's shared memory plus mmaping certain regions of the video device into clients' address spaces.

You can't draw shit using DRI.

Again: This neither KMS, nor DRI/DRM, nor OpenGL have anything to do with the braindeadness of "modern" X11 toolkits.

What does have something to do with it is reinventing the wheel and to all the rasterization work in software client side in code contained in the toolkit (Cairo in the case of GTK+, "raster" in the case of Qt). Why these custom rasterization engines have been implemented?

Well, mostly to be able to use the toolkits in an embedded environment that lacks a display server with drawing primitives (for example DirectFB). Eventually the scope expanded to draw single widgets for X11 and at some point it was decided that it would be easier to just render the whole thing in software and then blit.

People were dissatisfied with the graphics primitives offered by X11 / XRender. Yes, I admit, rectangles, triangles, polygons, lines, arcs and ellipses are not that much (/s); there's still the desire for Bezier curves. And yes, I admit that the X11 graphics model may be somewhat limiting when it comes to composing drawing layers, as you need it for, say CSS.

However I consider it mostly a lack of creativity and determination. XRender can be quite powerful if you know how to use it. And with Glamor at last it will be GPU accelerated.

digi_owldatenwolf9 years ago
And thats the thing. At least from my point of view, Cairo and the like was "happy" exiting within X11.

But once we got things like Compiz, a major shift in attitude towards X11 came about.

datenwolfdigi_owl9 years ago
> But once we got things like Compiz, a major shift in attitude towards X11 came about.

What the…? Are you trying to pull a "Hey, look over there, a three-headed monkey!"?! Why are you now pulling Compiz into the argument? It's totally irrelevant for this discussion.

Compiz is just a client that makes use of a concerted effort of the XRender, the Composite and the AIGLX extensions to composite whole windows to the screen. AIGLX is an acronym for "Accelerated Indirect GLX"; don't ask why that acroynm was chosen, because GLX itself already is accelerated and may be indirect.

Compositing also doesn't care about how you draw to a X11 drawable. All it cares about is: Here is a set of X11 drawables (windows, pixmaps), all server side, which you can draw to with server side accelerated rendering primitives. And then over there's a compositing surface (the composite layer window). The X11 server then is: "Now please give me drawing instructions on how to get the composite surface filled with the contents of the drawables!"

The compositing effects render completely server side, even if OpenGL graphics effects are used, thanks to indirect rendering command execution (heck, you can even compile the drawing commands in a display list (an ooold OpenGL feature; so old, it's been removed from modern OpenGL versions) and have the whole thing executed by sending a single glCallLists command to the X11 server via GLX).

digi_owldatenwolf9 years ago
Forest, trees, good night.
DonHopkinsdatenwolf9 years ago
There's no way X can do anti-aliasing, without a ground-up redesign. The rendering rules are very strictly defined in terms of which pixels get touched and how.

There is a deep-down irreconcilable philosophical and mathematical difference between X11's discrete half-open pixel-oriented rendering model, and PostScript's continuous stencil/paint Porter/Duff imaging model.

X11 graphics round differently when filling and stroking, define strokes in terms of square pixels instead of fills with arbitrary coordinate transformations, and is all about "half open" pixels with gravity to the right and down, not the pixel coverage of geometric region, which is how anti-aliasing is defined.

X11 is rasterops on wheels. It turned out that not many application developers enjoyed thinking about pixels and coordinates the X11 way, displays don't always have square pixels, the hardware (cough Microvax framebuffer) that supports rasterops efficiently is long obsolete, rendering was precisely defined in a way that didn't allow any wiggle room for hardware optimizations, and developers would rather use higher level stencil/paint and scalable graphics, now that computers are fast enough to support it.

I tried describing the problem in the Unix-Haters X-Windows Disaster chapter [1]:

A task as simple as filing and stroking shapes is quite complicated because of X's bizarre pixel-oriented imaging rules. When you fill a 10x10 square with XFillRectangle, it fills the 100 pixels you expect. But you get extra "bonus pixels" when you pass the same arguments to XDrawRectangle, because it actually draws an 11x11 square, hanging out one pixel below and to the right!!! If you find this hard to believe, look it up in the X manual yourself: Volume 1, Section 6.1.4. The manual patronizingly explains how easy it is to add 1 to the x and y position of the filled rectangle, while subtracting 1 from the width and height to compensate, so it fits neatly inside the outline. Then it points out that "in the case of arcs, however, this is a much more difficult proposition (probably impossible in a portable fashion)." This means that portably filling and stroking an arbitrarily scaled arc without overlapping or leaving gaps is an intractable problem when using the X Window System. Think about that. You can't even draw a proper rectangle with a thick outline, since the line width is specified in unscaled pixel units, so if your display has rectangular pixels, the vertical and horizontal lines will have different thicknesses even though you scaled the rectangle corner coordinates to compensate for the aspect ratio.

[1] The X-Windows Disaster: http://www.art.net/~hopkins/Don/unix-haters/x-windows/disaster.html

fulafelDonHopkins9 years ago
This is all true about the original core X protocol, doesn't prevent getting eg efficient antialiasing with protocol extensions. OpenGL over X and the RENDER extension are existance proofs that you can roll out new rendering systems in X.
DonHopkinsfulafel9 years ago
I was referring to "actually using the X11 drawing primitives in a clever way" and "why antialiasing was never enabled for X core drawing primitives", not adding an entirely new set of stencil/paint drawing primitives that don't interact with the existing ones, or changing the way X-core drawing primitives work to enable anti-aliasing.

If adding graphics primitives via X extensions worked as advertised, then X would have solved all these problems we're still having decades ago.

But as the HP technical support person said, "Why would you ever need to point to something that you've drawn in 3D?"

datenwolfDonHopkins9 years ago
> There is a deep-down irreconcilable philosophical and mathematical difference between X11's discrete half-open pixel-oriented rendering model, and PostScript's continuous stencil/paint Porter/Duff imaging model.

Indeed. But we're not talking about this. We're talking about the faux reasons why X11 toolkits started doing client-side rasterization (which is still pixel oriented), instead of doing just a little bit of work, making X-core primitives beautiful for framebuffers configured the right way.

> A task as simple as filing and stroking shapes is quite complicated because of X's bizarre pixel-oriented imaging rules.

Here's the thing: When XRender was added, it introduced a whole new set of framebuffer configuration options. It is these options that allow one to specify antialiasing modes and it's the very same options that also have to be used if one want to use OpenGL multisample antialiasing on a window facing framebuffer (without a multisampled window format the only way to get MSAA with OpenGL is to create a multisampled OpenGL framebuffer object render-target and perform multisampling resolution via framebuffer blitting).

The point is: It would have been trivial and perfectly allowed to alter the rasterization specification for X core drawing primitives in case a multisampled framebuffer configuration was active for the window. Replacing drawing behavior is something perfectly normal when introducing new OpenGL features (either by new version or extension) and the very same would have worked just as fine for X core.

    if( fbconfig.multisampled ) {
        draw_Xcore_primitives_with_new_antialiased_rules(...);
    } else {
        draw_Xcore_primitives_with_traditional_rules(...);
    }
no existing legacy application would break by such extension.

Note that if you want to replace X11 with something sensible, you're preaching to the choir. I've been advocating the use of abstract display systems, that are graphics primitive oriented (instead of pixel oriented) for years (actually over a decade now). But I've also found that many proponents and opponents of X11/Wayland/etc. do not actually understand the problem at all.

DonHopkinsdatenwolf9 years ago
Well the thing is, there's a funny story behind that rendering code:

All of that beautiful free open source rendering code was originally conceived in the master bedroom of its parent's house, the X11 server. Its parents had ambitious but traditional plans for it, from the time it was only one bit deep, and as it grew up and learned how to walk around on eight bits with the help of a colormap.

Then when it finally reached puberty and learned how to composite 24 bit images into deep visuals, it needed more space and some privacy of its own, and room to store its growing collection of pixels, so its parents moved it into the X Rendering extension in the basement.

But it still had to share a single telephone line with its parents, who monitored all of its communications and screened its friends, while insisting it do all its chores like taking out the garbage and cleaning its buffers, before going out and playing with its friends.

It managed to open up a shared memory window into the back yard, so it could move larger quantities of pixels and cache back and forth between its local clients and its room in the basement, without waking up its parents in the middle of the night.

But as it grew up, it started dealing more and more little packets of pixels with its local clients on the side, and going away on frequent rides to make deliveries to remote clients, so its parents became suspicious and angry.

This caused many problems, but managed to work for a while. But then when it got older, it started getting invited to sleep over parties, where its friends and growing list of clients invited it to bring all its favorite pixels to their house to share and mix together.

Eventually it was regularly performing as Master Compositor at huge parties, pumping out colorful house pixels with throbbing techno alpha channels, deep into the night and on into the early morning.

But that behavior caused terrible problems with its parents at home, who didn't approve of being out of the loop, but still having to foot the bill for its Frosted Flakes and AOL subscription.

Finally it got its shit together enough to move out of its parent's basement, and into a small shared library apartment in Cairo, so it could perform freelance jobs for its growing list of clients who didn't want any involvement with its parents. Now it could party with its friends and clients, without its parents knowing and interfering with its pursuit of happiness.

Its parents had some very old-fashioned parochial ideas about font rendering and naming. They firmly believed that a font's place was in the pixmap, angrily hated the ideas of mixed alpha antialiasing or same alpha transparency, approved of the missionary transform but were otherwise affine-transphobic, and they sternly disapproved of outline fonts with soft voluptuous curves and gentle antialiased edges.

But with its new found maturity and hard earned freedom, it got lucky enough to hook up with a sexy outline font rendering stripper named Crystal FreeType and an elegant internationally suave text layout pole dancer named Paris Pango. As a free software threesome, together they integrated seamlessly, worked beautifully, hooked up with prolific scripts and toolkits, and painted some truly epic scenes.

As their careers advanced, they were employed in many important high profile jobs, including as Art Director of the renowned GTK+ Disco Dance Party (the Soul Train of the Desktop, syndicated to a long list of platforms and scripting languages), and MC (Master Compositor) of the internationally televised Mozilla Firefox Web Surfing Variety Show (based on the original Solid Gold Netscape show, that inspired all the others).

Not only did they cover other famous artist's works like HTML, CSS, SVG, PDF and XUL, but they also showcased their own instant-classic Canvas API, which revolutionized the world wide web and matured from a popular fad into an essential institution used by everyone around the world.

So at this point in history, there's really no point in asking such successful code to go back and live in its parent's basement again, because its style would be cramped, and nobody would want to visit it there, now that it finally has its own rich interesting life to live in the real world. ;)

datenwolfDonHopkins9 years ago
> So at this point in history, there's really no point in asking such successful code to go back and live in its parent's basement again, because its style would be cramped, and nobody would want to visit it there, now that it finally has its own rich interesting life to live in the real world. ;)

But like all stories, also this one has a twist for the dark: Although the art created by the protagonist was of high quality, the artists technique was behind the curve of what people actually wanted to see. The sad thins is, people didn't exactly know what the wanted to see, unless it was shown to them. Also our protagonist artist had a obsessive preference for his drawing substrates: The only canvases he'd be drawing on would consist of orthogonal grids and would only fill each cell with a solid color. His sidekick Crystal Freetype was okay with it, but also figured, that some people, for some reason liked it, if she tripled the resolution of the grid in one direction and biased the hues by a periodic red-green-blue pattern. But she never got really perfect at it.

Some say, and those would clearly be right, that the reason for that was, that the people who'd wanted to see their art would do it only at galleries: But those galleries had problems with reproducing their illumination. Each gallery had slightly different light. For a long time each gallery tried to compensate with its own tools, documenting what had to be done in a customary way. Eventually all these galleries got their act together and agreed on a specific way to tell each other and the artist how the light looks at their place.

They also realized that for maximum effect the light at the gallery had to be taken into account when drawing the pictures.

Unfortunately our hero "rendering code" never properly learnt what to make out of the information he got from the gallieries. He preferred to work with his preferred colours Red, Green and Blue and knew how to mix them according to the color theory on RGB colors "s" as laid down by his idol "Adobe" (also called sRGB), but Crystal FreeType never got the knack for it, messing up the Gamma of her antialiasing. There would have been of course the color theory of linear contact color spaces, but our hero abhorred the idea to work with "imaginary" colors, or that the dyes in his paints would only look right after being treated with a colorspace-converting developer.

Sometimes two galleries would want to exhibit the same picture. Or one gallery one part and another gallery the other part; or maybe three or four libraries. Of course our hero could have carefully planned how to draw each part of his paintings, but this would take time and more importantly much more extra effort in dealing with his clients. So instead most of the time he'd just draw for the lowest common denominator. The pictures would be not as pretty as they could have been, but at least everybody got to see something.

But what really dragged down our hero was, that his paintings were heavy. Moving them around in towns where his clients lived already was hard work, but it was manageable, thanks to the heavy dutry transport system available. But sometimes a gallery away from his clients wanted to show the pictures and then it got really messy.

Some clients remembered that when "hero rendering code" still lived at his parents, he could give his parents a call and place an order for a picture to be drawn. He'd just telephone what it should look like, which went rather quickly. Of course some clients adopted a inefficient way of placing their orders, stating one item, asking if it was on the list then asking for the next one, and so on. Other clients became clever and just placed their whole order before asking back; often "hero rendering code" would already be at work, when the confirmation on the order arrived. Of course there would have still be problem of moving the picture to a gallery… but as it turns out the parents had a huge library with reconfigurable walls at their house. So clients who'd just want to show drawings drawn by "hero render code" at his parents libraries according to their telephone calls could do this from practically everywhere, even over telephone lines that could barely carry a conversation, yet patrons would see the same image quality.

But now with "hero rendering code" insisting on doing his work in the clients town transporting the pictures became a major issue. Some clients found the services of the Shrinker Codec Union; their union workers would deconstruct the drawings of "hero rendering code", sweep things under the rug they's presume the patrons would never see and transport that. For very complicated images they figured out, that this was actually faster than trying to talk the complicated picture through the telephone line; yet the price was, that pictures often suffered in their colour brilliance. It was however Crystal FreeType who's be extremely outraged, because it was often her expertly crafted contributions which got mauled by the Shrinker Coded Union workers.

Meanwhile new drawing tools appeared on the market; or rather they've been on the market for quite some time, but hero rendering code never learnt how to peoperly use them. His parents added an annex to their house though where those drawing tools could be used. But often it were hero rendering codes clients who'd then use these tools for their own means. Hero rendering code often tried to get to grip with those tools but often failed miserably; maybe hell never learn. And Crystal FreeType, she doesn't even understand how to use these tools properly; FreeType lais out her pictures and spans of lines, she'd colour in particular colors. But those new tools, if they are used like that produce poor results or it takes much longer to get a nice picture of a character, than using the old tools. But it's not that this is impossible. New artists who are expert in the use of these new tools have become runners up.

But many of hero rendering code's clients are now so closely tied to him, that they can't make use of those new expert drawers services. And they remember the times, when they could give the parent's "X11" a house call and have them write down the order for the picture.

Wouldn't it maybe be possible to create a lodging home and gallery for all these new artists in town, where one could place house calls for showing their drawings on premises? In a Gallery where the artists can paint the interesting parts of their painting in the room where they're going to be shown, matched to the lighting conditions then and there? Without the need to haul havy paintings over land?

to be continued…

theohtracker19 years ago
Isn't it essentially possible for anyone with the equivalent of shell access on a remote machine to implement a system like Rob Pike's Sam editor? You can roll your own economical protocol, which could include mobile code fragments (regexps sent to remote process for execution, for starters). I'm not sure why you think we're stuck with the browser for this, or if that's a bad thing for most applications that don't involve something bandwidth-intensive like realtime 3D.

"Sam is implemented as two processes connected by a low-bandwidth stream, one process handling the display and the other the editing algorithms. Therefore it can run with the display process in a bitmap terminal and the editor on a local host, with both processes on a bitmap-equipped host, or with the display process in the terminal and the editor in a remote host. By suppressing the display process, it can even run without a bitmap terminal."

http://doc.cat-v.org/plan_9/4th_edition/papers/sam/

mcguiretheoh9 years ago
"You can roll your own economical protocol..."

Aye, and there's the rub.

0xdeadbeefbabemcguire9 years ago
I don't know:

"I’m doing a (free) operating system (just a hobby, won’t be big and professional like gnu) for 386(486) AT clones. This has been brewing since april, and is starting to get ready. I’d like any feedback on things people like/dislike in minix, as my OS resembles it somewhat (same physical layout of the file-system (due to practical reasons) among other things)." -http://www.thelinuxdaily.com/2010/04/the-first-linux-announcement-from-linus-torvalds/

theohmcguire9 years ago
Maybe so. Actually I think maybe the way you factor the application may be more crucial. I love the story about MVC being invented in the context of shipbuilding, because it's such a heavy duty business, and obviously the view (drawings) and the model (the state of steel fabrications under construction) have such totally different characteristics with respect to changes.
e40tracker19 years ago
though, I will admit RDP is pretty serviceable, X11 just seems too janky to work over a really remote connection... and VNC, well it works, but, sigh...

I used, every single day, a product called "FastX" from StarNet, that runs X over RDP. It means the X programs I display on my Mac (which are running on a Linux box) are disconnectable/reconnectable. I disconnect when I'm ready to go to work, reconnect there, and do the same thing when I finish at the end of the day.

Absolutely lovely.

tracker1e409 years ago
Again, commercial product, low-end/individual pricing doesn't seem to be listed... There have been a few similar products about a faster remote experience with linux. I've tried a couple, but the commercial bit always rubbed me a little bit... I mean, if the pricing is open for personal use, and reasonably priced, I'm all in.

That hasn't been the case in my experience.. thanks for the heads up though. I tried NoMachine's NX for a while, which was very nice.

tracker1tracker19 years ago
Okay, looks like NoMachine is now free for desktop/personal use all around... didn't use to be the case (outside linux version)... May have to give it another look.
simula679 years ago
This is the project the demise of which lead to Java : http://www.blinkenlights.com/classiccmp/javaorigin.html
DonHopkinssimula679 years ago
Keep in mind that while extremely fascinating and enlightening and well worth reading, and how apt his comparison to the Gaza strip, that essay is from Patrick's own unique perspective, and he does not necessarily share all the same opinions as the dispassionate people for whom he recommended euthanasia. ;)
arethuza9 years ago
I used HyperNeWS for a few years on a project and had great fun with it (front end in PostScript/NeWS, back end in Common Lisp and plumbing in C).

You could do some neat tricks - draw something in the graphical editor, copy it and paste it as the shape of a window (stack). Another thing you could do was log in through the PostScript shell to the server, navigate to your window (or anything else) and fiddle with it using standard PostScript 2D transforms. e.g. You could gradually rotate a terminal window, although I would never have done this to a colleague as they were using it, honest.

I rather enjoyed working in PostScript although I believe there was a C to PostScript compiler built at one point (inevitably this being done in Glasgow it was called pdb for "pure dead brilliant").

Don't think there was much of a security model in NeWS so probably not a great fit for today's world but it was so much cooler than X.

mhandley9 years ago
NeWS was really cool. Back in 1987/88 I used an early release of NeWS for my final-year undergrad project. It was made a little more challenging by only having a photocopy of a pre-release of the documentation, that was missing all the diagrams.

The model was really cool - completely different from X11, which is much more limited. With X, your app runs on the client and sends drawing commands to the display server. With NeWS, you can split your app arbitrarily between the client and the display server. The client simply extends the server by sending it postscript functions to locally interpret when asynchronous events happen. So you can write new widgets, and they run entirely on the display server so get really low latency, while the rest of the app is running remotely over a piece-of-wet-string network connection.

You could be sitting in from of a display running a nice interactive GUI, with the app running in a different country or over a modem connection. At least you could if the application designer got it right. Getting the split of functionality between the client and server (and the communication between them) right was definitely more challenging than writing X apps, but the benefits were significant.

arethuzamhandley9 years ago
Speaking of documentation - it was rather cool that when you purchased a Sun at that time (or maybe it was when you purchased OpenWindows) you can a set of official Adobe PostScript books (blue/red/green?).
mcguiremhandley9 years ago
8 1/2, from Plan 9, used a similar architecture, IIRC, because of the smart terminals it was written for. Check out the editor Sam for an example; one side is ed++, the other is the GUI.
mveetymcguire9 years ago
You mean the Blit terminal. The Plan 9 window managers use a different architecture that takes advantage of Plan 9's design. Both are from Rob Pike though so they look basically the same.
jonjackymcguire9 years ago
There are emulations of the Plan 9 smart terminals that run under Unix-like operating systems. They are quite different from xterms - they have much simplified communication with the remote host but much richer local editing capabilities:

http://sydney.edu.au/engineering/it/~matty/9term/

https://github.com/chneukirchen/tt

The original Plan 9 terminal and window manager are described here:

http://plan9.bell-labs.com/magic/man2html/1/rio

digi_owlmhandley9 years ago
Sounds like what you get if you ran HTML5+CSS+JS as a desktop...
bausdigi_owl9 years ago
I've thought exactly the same thing. Gosling's NeWS was way ahead of its time and we ended up getting there again sort of by accident with the web.
digi_owlbaus9 years ago
I dunno how much of it was by accident.

I get the feel it was the aim of Netscape from the virtually the beginning.

After all, the browser war was a proxy war for the server market and the intranet concept. As in using internet technology for the corporate LAN, with the web as the GUI for corporate servers.

david-given9 years ago
NeWS was lovely --- the application got to download custom logic via Postscript onto the display server and run it there, so you could run interactive applications quite comfortably on high-latency links. Proper MVC separation!

Also, it had PizzaTool. http://www.donhopkins.com/home/catalog/images/pizzatool.gif

Source code: http://www.donhopkins.com/home/code/pizzatool.ps.txt

I've always rather wondered why no open source NeWS clone never came along.

arethuzadavid-given9 years ago
Do you think the forking off multiple processes to draw all of the toppings concurrently was premature optimisation? ;-)

[Edit: I was pleasantly surprised at how readable that code is]

david-givenarethuza9 years ago
Possibly premature, definitely not optimisation...

...but I saw it running in real time, and it was actually pretty cool to see all the toppings appear (this was on a Sparc ELC-class machine, so it didn't have much grunt). Seamless dithering on the monochrome display, too.

Also, the spinning the pizza was done by rotating the rendered bitmap. The inaccuracies meant that as you spun it, the toppings would all gradually blend into each other. I suppose that's realistic, in a way.

DonHopkinsdavid-given9 years ago
I'm glad you had fun! Thanks for the mention. The point of PizzaTool was to be well commented example code for developers to read, that showed how to plug many different parts of The NeWS Toolkit together (thus all the drag-n-drop and threading and image transformation stuff).

You're right, that was the best "melting" effect I could come up with, within the constraints of the PostScript imaging model. PizzaTool spins the pizza automatically, because I got tired of turning the image rotate tool in Stuart Marks' RasterRap by hand again and again to get that melting effect. I left it running over Christmas vacation, and it collapsed into an blob. But you can still add more toppings at any time while it's spinning!

Here's a demo of spinning pizzas, dragging and dropping images between RasterRap and PizzaTool, and cleaning the pizza tray afterwards with a wave. Sorry about the bad video quality!

https://www.youtube.com/watch?v=avJnpDKHxPY&feature=youtu.be&t=21m42s

david-givenDonHopkins9 years ago
While you're here...

What was the NeWS security model like? Could it safely combine Postscript programs from multiple users on the same display server without information leakage between users?

And I suppose the NeWS source code got eaten by lawyers long ago and is unlikely ever to see the light of day?

DonHopkinsdavid-given9 years ago
NeWS was just as "open" (in the bad security sense of the term) as X11 was at the time, and didn't have any protection between clients, but was much more reflective and easier to explore. Once you're in, you're really really in.

Tools that imported EPS could be tricked into doing all kinds of stuff like forking Unix processes, and open files. And being able to open files also meant opening sockets with the %socket syntax.

NeWSPrint was a laser printer that ran NeWS on a workstation to render pages, in a stripped down presumably firewalled environment. But I was able to cause an error in a certain context that left an execution stack dump on the operand stack before calling my code, and inside that stack dump were nested executable arrays of PostScript code that contained an inlined unrestricted 'file' operator. So it was possible to make a PostScript file that would connect to the finger server and print out a list of anyone who was logged in. Or anything else if you can fish out a fork operator!

The NeWS source code is in that indeterminate "not lost but not published" state.

It would be a lot of fun and not so difficult to reimplement clean room NeWS in JavaScript from scratch, using canvas for all your PostScript rendering needs, which would otherwise be the hard part. James Gosling wrote the original SunDew interpreter in a weekend, but the graphics stuff took years to get right.

Gosling's SunDew paper: http://www.chilton-computing.org.uk/inf/literature/books/wm/p005.htm

A few years ago I dabbled in writing a spec and some plumbing for NeWS in JavaScript, but haven't had the time to work on it recently. If anyone wants to work on it, I'd be happy to answer questions and put a little more time into it when I can.

Source code: http://donhopkins.com/home/sundew/sundew.js

It doesn't do anything exciting or graphical yet, since I was just working on the interpreter. So far I've classified and listed out all of the operators you'd need to cover, made nucleus classes for the vm, processes and objects, modeled shared bodies for dicts, array and string sub-intervals, stubbed out all of the NeWS primitive and magic dictionary data types, and stopped just short of writing the main interpreter loop in PSProcessBody.run, which will work a lot like ps.ps: http://donhopkins.com/home/code/ps.ps.txt

It still needs a lot of fleshing out, including a tokenizer (but that's trivial for PostScript) and many operators and graphics (but that stuff's fun!).

I'm sure that with some more work on a clean room JavaScript NeWS server, it would be both possible and legal to run a lot of the old NeWS 1.1 demos and apps like psiber, and maybe even later OpenWindows stuff like pizzatool, pretty darn fast in a web browser.

mhd9 years ago
Ah, pie menus... Weirdly enough, the only place I see them these days is games.
jle17mhd9 years ago
That may be due to Don Hopkins. He did the pie menus for NeWS and latter for The Sims.
shrizzajle179 years ago
jwcad usage is very piemenu-centric.
agumonkey9 years ago
Maybe my google-fu was blurred by the 'news' term but it was very hard to find things about it (beside "NeWS was great").
baus9 years ago
NeWS wasn't the only windowing system to use Display PostScript. The other big one was NeXTSTEP which obviously became OS X.
DonHopkinsbaus9 years ago
NeWS was not actually Adobe's Display PostScript, but it was Sun's independent implementation and specialized dialect of PostScript, supporting light weight processes, overlapping arbitrarily shaped canvases, window management, event distribution, garbage collection, networking, object oriented programming, etc.

The most important ability that NeWS had, but was missing from Display PostScript and its successors (OS/X Core Graphics, PDF, SVG, canvas API, etc), is the ability to download code to create an efficient custom high level application specific protocol between the client and server.

That essential ability is what people call "AJAX" these days, now that PostScript has been supplanted by JavaScript and a whole bunch of different APIs, and now we're even downloading shaders to the GPU! Truly exciting!

James Gosling chose PostScript from the start, for how its network programming ability dovetails with its graphics and data representation, instead of nailing it onto the side of a bunch of different technologies as an afterthought.

To quote the comparison from the wikipedia article:

NeWS was architecturally similar to what is now called AJAX, except that NeWS coherently:

1) used PostScript code instead of JavaScript for programming.

2) used PostScript graphics instead of DHTML and CSS for rendering.

3) used PostScript data instead of XML and JSON for data representation.

DonHopkinsDonHopkins9 years ago
Here's a deeper discussion of the technical design differences between NeWS and Display PostScript, from a discussion with Robin Schaufler and James Gosling:

    From: James Gosling <jag@Sun.COM>
    Subject: Re: Display PS vs NeWS
    Date: 22 December 1988 at 19:35:08 GMT+1
    To: Robin Schaufler <robin@Sun.COM>
    Cc: don@brillig.umd.edu (Don Hopkins), sevans%norquay@Sun.COM
Robin: Don, I'm replying just to you, not to NeWS-makers (trying to stay out of trouble). The one major area where NeWS cannot emulate Display PS is that of memory management. Display PS introduces the notion of spaces; effectively, each process group gets its own heap. Kill the process group - the memory gets cleaned up whether it has cycles in it or not. Very nice, actually. But a major project for us if we decide to adopt it. Fortunately, we already started on a new memory management system for data localization that will help some with implementation.

James: The thing that we don't have is a thousand caveats on what can be assigned to what. You can't have pointers that cross spaces in Adobe's scheme, so as you read through their documentation you find zillions of restrictions that make life very difficult. Also, the "kill the process group..." doesn't really apply: they don't have process groups. Each process is alone. If you want to share, you have to stick stuff in the "shared" space or be forked from a common parent. save/restore has bizarre semantics in this memory model: eg. if a process executes "save" then any other process that shares the same storage pool is blocked (honest, I'm not making this up!)

Robin: We also can't emulate their lock mechanism. I had the distinct impression that they analysed NeWS monitors completely, figured out exactly what was wrong with them, and fixed it. I really like this extension, but again, considerable work for us (tho less than for the memory management extension).

James: Far from considerable work and of dubious value: almost no one has ever used locks. DPS locks are identical to NeWS monitors (they're even applied with the same primitive "monitor"). NeWS is missing condition variables, which are a good idea but not difficult to implement. Events are the common form of process syncronization in NeWS.

Robin: Another area we would have trouble emulating is their new halftoning operators, but that seems like less of a problem. The new operators allow for tiling, which we've implemented for X, but we'd still have to hook it up for NeWS, and provide their model, which isn't quite the same model. Actually, NeWS probably can emulate it but it would be beastly slow.

James: Actually, the DPS threshhold are NOT the tiling operators that PostScript so badly needs. If you read it carefully, for devices with more than one bit per pixel you can't get the tiling effect. Halftones continue to only be useful for halftoning.

Robin: In a few cases they picked the same names and operator spec as us, but in general they didn't try to be compatible. A few incompatibilities seem gratuitious to me.

James: eg. processes

grymoire19 years ago
I used and wrote code for NeWS - the pre-curser for Java. I was also at the talk where Gosling admitted the problems it had. In some cases, a lot of features of Java were in response to the problems with NeWS.

NeWS was so awesome when it came out, where you can dynamically change the interface at any time. But NeWS had some horrible bits. As a language, it was very hard to read. If you saw a name, you didn't know if it was a variable or a function. That is, you didn't know if the value would be pushed onto the stack, or if it would pop some values off the stack. Best Practice was to add a comment after each line to indicate what the top of the stack would look like after that line. So if you got lost reading one line, you could figure out what the next line would do. But without it, after seeing three words in a row that you didn't fully understand, you were hopelessly lost.

NeWS (PostScript) was a write-only display model. You could not remove anything painted on the screen. You had to write over it. Adobe had Display PostScript, but that never took off.

And because of the write-only display, NeWS had a hard time freeing resources. Garbage collection was very difficult because of this. You would allocate a white square, and never truly "remove it" from memory. As I recall, a page refresh had to re-interpret all of the code - there wasn't any caching of the window.

Also - the entire display model with postscript-based scaling was CPU intensive, especially if you had a computer with slow graphics (quite common at the time).

DonHopkinsgrymoire19 years ago
PostScript uses an immediate mode state machine, kind of like GL, but a lot simpler. So there is a "graphics state", and while it contains a reference to the current font and accumulates a path as you draw it, there was nothing like a display list or object oriented graphics that would remember to draw a square and never get garbage collected.

Laser printers typically had a linear heap based memory allocation system that let you ensconce and roll back all modifications on a page with save/restore, but NeWS never used that for anything practical (although it did support it in a weird inefficient way of recording and reverting all mutations, which was more for backwards compatibility so the PostScript previewer could rendering existing PostScript documents that used save/restore, and never used in any interactive user interface toolkit or app outside a PostScript previewer).

Instead of a heap like a laser printer, NeWS used a reference counting garbage collection system, so once all references to an object were gone, it and any non-reachable sub-objects would be destroyed.

To work around problem of circular references leaking memory, later versions of NeWS supported soft reference counts: each object would have both a soft and a hard reference count, and you could soften a reference if you wanted to hold onto an object but be notified when it was no longer otherwise accessible. Once all the hard references to an object went away, if the soft reference count was still not yet zero, it sent out a /Destroy event on that object which anyone holding a soft reference to it must be listening for, and they clean up their reference in response to, and it's only deleted once its soft reference count goes to zero. It's a weak reference [2], but it used NeWS's built-in event distribution system to notify reference holders (since there was no way of PostScript code to hook into the destroy callback of an arbitrary dictionary).

Needing to use soft references directly wasn't very common in ordinary NeWS programming, and most uses were wrapped up behind the scenes by the toolkit (like the selection service listening for the selected object to be destroyed, text fields and menus pointing at each other for focus and pinned menu management, etc).

NeWS canvases could have a backing store that you could enable or disable, and they could be retained off screen, and top level windows were usually retained. Technically it was just a hint and you always had to be prepared to repaint in response to a /Damage event, but all servers supported retained canvases, and it still made sense to turn off retained mode in the case of simple efficient graphics on 24 bit color displays.

The art was in coming up with simple efficient ways of representing the data and generating the code doing the drawing in the server, so it could draw as fast as possible. Cool things were possible by dynamically generating PostScript code with inlined numbers, colors, fonts and other invariant parameters, in response to window resizing or layout changes and higher level data. You could inline calculated values into stupid-but-fast specialized instance methods (NeWS used a prototype based object system, that let you promote methods into instances), by using the PostScript equivalent of Lisp's "backquote" macro. [2]

[1] Weak References: https://en.wikipedia.org/wiki/Weak_reference

[2] TNT Slider Design Document: using PostScript "backquote" macro to cache efficient rendering code: http://donhopkins.com/home/archive/NeWS/tnt-sliders-design-2.txt

rthillegrymoire19 years ago
"Adobe had Display PostScript, but that never took off." Well, except for it's use in NeXTStep. Though I think they threw it out in favor of PDF/Quartz for OS-X.
mozumder9 years ago
OpenLook window manager was great.. I loved how customizable it was, using things like text files to customize root menus. It was also the best looking WM I've used.