tag:blogger.com,1999:blog-188414702024-03-13T21:42:43.815+00:00York Haskell CompilerUnknownnoreply@blogger.comBlogger37125tag:blogger.com,1999:blog-18841470.post-62565051835329512992011-04-03T19:20:00.004+01:002011-04-03T21:52:38.446+01:00Yhc is deadSomeone recently asked on the <a href="http://haskell.org/haskellwiki/Yhc">Yhc</a> mailing list if Yhc was dead. The answer is yes, noone has been working on the compiler for several years. The <a href="http://www.haskell.org/nhc98/">nhc98</a> compiler, from which Yhc originally forked, is still maintained. Since this is the end of Yhc, I thought I'd share a few random thoughts about Yhc:<br /><br /><ul><br /><li>Writing a Haskell compiler is a big undertaking, and the work required to compile a moderate number of programs from <a href="http://hackage.haskell.org/packages/hackage.html">Hackage</a> is immense. Too many libraries rely on something <a href="http://haskell.org/ghc/">GHC</a> specific - either language or runtime features. The GHC team have gone out of their way to ensure that their compiler is by default a standard Haskell compiler, which has kept Haskell as a viable language separate from GHC. But without widely-used competing implementations, programs tend to end up depending on GHC.</li><br /><li>There are still other non-GHC Haskell compilers, and I wish them all well. Many offer features missing in GHC (compile to Javascript, tiny runtime system, extreme portability, code mobility etc.) - there are lots of interesting ideas floating around.</li><br /><li>Yhc started in a fairly haphazard way, and became a fork of nhc long before anyone had noticed. Had Yhc's contributions been rolled back into nhc they'd probably have had more lasting impact.</li><br /><li>One of the big draws for Yhc was it's ability to take a Haskell program, and produce a Core complete program. Despite all of Yhc's other weaknesses, Yhc Core drew several people to the project - I hope something equivalent appears for other Haskell compilers.</li><br /><li>If you don't understand monads, you aren't yet ready to write a Haskell compiler.</li><br /><li>If you have a group of Haskell programmers in the same place, you should try a group project. It's fun.</li><br /><li>The biggest challenge for Yhc was the build system - we ended up with 10,000 lines of Python Scons scripts. Without a robust build system nothing else matters. When our sole Python hacker left the team that was the beginning of the end.</li><br /></ul><br /><br />Working on Yhc was fun, and I learned a lot. Thanks to everyone who was involved.Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com3tag:blogger.com,1999:blog-18841470.post-54028810020876424832008-05-18T20:11:00.009+01:002008-05-19T05:15:15.498+01:00Haskell on BEAMsRecent developments showed that Haskell, via conversion from Yhc Core, can run on:<br /><ul><li><a href="http://haskell.org/haskellwiki/Yhc/Pyhi">Python runtime</a><br /></li><li><a href="http://haskell.org/haskellwiki/Yhc/Javascript">Javascript in a web browser</a></li><li><a href="http://www.cs.york.ac.uk/%7Emfn/reduceron2/">Reduceron</a>, which is more hardware than software<br /></li></ul>Ability to run Haskell on some runtime gives Haskell programmers access to some or all features provided by that runtime, while keeping the features provided by the language unchanged. So it may be worth trying more runtimes to run Haskell programs on.<br /><br />BEAM is a virtual machine Erlang runs on. Erlang provides a distributed fault-tolerant concurrency-oriented computing platform, and its use keeps widening for various kinds of applications. BEAM is in plural in this blog post's title because a distributed Erlang program is running on multiple instances of BEAM, and so would a Haskell program in the same environment.<br /><br />There are projects aimed at compilation of some programming languages to Erlang or Core Erlang:<br /><ul><li><a href="http://code.google.com/p/erlyjs/">Javascript</a></li><li><a href="http://code.google.com/p/erlocaml/">Ocaml</a></li><li><a href="http://forum.trapexit.org/viewtopic.php?t=14923">Lisp</a></li><li><a href="http://blog.tornkvist.org/blog.yaws?id=1190846785574003">Subset of Haskell syntax</a> (Haskerl)<br /></li></ul>So, why not add a compiler from Yhc Core to both collections?<br /><br />For about a month I have been experimenting with a converter of Yhc Core to Core Erlang. The converter was in part based on the Javascript backend converter, and in part on Haskerl. The goal was set to run a Haskell program that utilizes some commonly used subset of Haskell Prelude, and is also able to deal with creation of Erlang processes, sending, and receiving messages. <a href="http://www.erlang.org/doc/getting_started/conc_prog.html#3.2">This program</a> served as an example.<br /><br />The goal was reached, so I would like to publish the results for everybody to see and criticize. Any feedback is appreciated.<br /><br /><a href="http://www.haskell.org/haskellwiki/Yhc/Erlang/Proof_of_concept">Read on</a>.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://bp1.blogger.com/_NNZwTv8k1kM/SDDnTnXdc0I/AAAAAAAAAAM/zG_fHTXBEJs/s1600-h/hs_beams.jpg"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer;" src="http://bp1.blogger.com/_NNZwTv8k1kM/SDDnTnXdc0I/AAAAAAAAAAM/zG_fHTXBEJs/s200/hs_beams.jpg" alt="" id="BLOGGER_PHOTO_ID_5201911893783835458" border="0" /></a>Dimitry Golubovskyhttp://www.blogger.com/profile/10316900314693187190noreply@blogger.com2tag:blogger.com,1999:blog-18841470.post-3746698385264848182008-03-28T18:23:00.016+00:002009-04-27T18:08:08.205+01:00Yhc/Javascript backend: personal/historical notesIt looks like this blog has been silent for more than a year ;) In the light of the first milestone reached recently by one of Yhc subprojects, the Javascript backend, I am happy to break the silence and put some personal historical notes as well as notes about the current status of the project I have been working on for more than 1.5 years.<br /><br /><h2>1. The "crazy idea"</h2><br />I was not definitely the first to have a thought to convert Haskell to Javascript. One that I can now recall was a piece of text on this <a href="http://www.updike.org/articles/The_deal">page</a> (by Jared Updike, dated March 2006):<br /><pre><br />------------------<br />...In addition I was thinking it would be cool to compile Haskell to<br />JavaScript so then you could actually write arbitrary things in<br />Haskell and run them in browsers...<br />------------------<br /></pre><br />There was a number of others that I lost track of. Anyway, the idea was in the air, and seeing other people expressing it prompted me that it was worth trying.<br /><br /><h2>2. First experiments</h2><br />I started my experiments with Nhc98 in June 2006. The intermediate form of Haskell program before it was turned to bytecodes was called PosLambda. It was not very easy to understand it, but some primitive things worked out. Later I noticed that the<br /><a href="http://haskell.org/haskellwiki/?title=Hajax&oldid=5361">Hajax</a> page appeared on Haskellwiki (the link points to its earliest version) which referred to some of my developments. I started my own Wiki page, <a href="http://www.haskell.org/haskellwiki/STG_in_Javascript">STG in Javascript</a>, sort of response to the Hajax page. The "STG in Javascript" page is still there, showing the progress during that time (Summer 2006). Later, another developer (nickname Vir) picked up the topic, however his work is completely independent from mine.<br /><br /><h2>3. Yhc core</h2><br />Some time in September 2006 I learned (thanks Neil) about Yhc Core: an intermediate format that Yhc produces during compilation, similar by its idea to GHC Core, but much simpler and with more stable format. Plus, it has an externalized form, so in order to process it there was no need to alter the code of the Haskell compiler itself. These benefits were very attractive, and Yhc Core had much in common with PosLambda, so transition of my converter was relatively painless. In October 2006, the sources of the Javascript converter were checked into the Yhc darcs repo, so the Javasscript backend "officially" became a subproject of Yhc.<br /><br /><h2>4. The Roman echo</h2><br />On November 16, 2006, the first <a href="http://darcs.haskell.org/yhc/web/jsdemos/Echo.html">runnable example</a> was published. It was a web page with an input field. When a decimal or Roman number was typed in, it responded with the same number converted to Roman or decimal (conversion code by Malcolm Wallace), plus approximate time that the conversion took. First results were perhaps terribly slow: time shown might be seconds, yet it worked.<br /><br /><h2>5. DOM interface</h2><br />It was absolutely necessary to find a way how to call <a href="http://www.w3.org/DOM/">DOM</a> functions from Haskell. In the first Roman echo example, these calls were manually coded, but for more than one hundred of them defined by the Web Consortium, that did not seem acceptable. In addition, accurate representation of DOM methods' argument type information was critical, as one of Haskell's benefits in Web application design is strong static typing (as opposed to Javascript with no static typing at all).<br /><br />Happily, the Web consortium provides DOM interface specifications in <a href="http://www.omg.org/gettingstarted/omg_idl.htm">OMG IDL</a> syntax. And there is an utility that is capable of understanding DOM IDL: <a href="http://www.haskell.org/hdirect">H/Direct</a> (by Sigbjorn Finne). The way H/Direct reflected IDL interface inheritance was simple, but not very open-ended: all sub-interfaces of an interface were presented as constructors of an algebraic data type, which meant that in order to add an instance of an interface, the whole thing needed to be recompiled as Haskell does not have extensible algebraic types.<br /><br />So, the the Haskell code generator was stripped off from original H/Direct code while the IDL parser remained in place. The new Haskell code generator transformed the hierarchy of IDL interfaces into hierarchy of Haskell type classes. This is a more open-ended solution, as instances to classes may be defined elsewhere without changes to class definition itself.<br /><br />Finally, DOM IDL interface definitions downloaded from the Web Consortium website were passed through the modified H/Direct, yielding tens of modules (each DOM interface was placed in a separate module), preserving the type information as much as it was possible (indeed, were DOM interfaces designed with much regard to strong static typing anyway?).<br /><br />Of course, some manual coding still was necessary (mainly around the type class defaulting problem), but its amount is incomparably smaller than the amount of Haskell code generated from the DOM IDL definitions.<br /><br /><h2>6. Unicode</h2><br />Surprisingly, Javascript code running in Web browser has almost no access to Unicode character properties. Case (upper to lower, and the opposite) conversions are of course available, plus something can be done using regular expressions, but that's mainly it. No direct access to Unicode character category, no titlecase conversion, etc. Some logic was borrowed from Hugs and GHC where such information was provided as a lookup table (by character code ranges). This resulted in about ~70 kbytes added to the Javascript runtime included in every Web page generated, but at least essentially the same Unicode character properties API that is available in Hugs and GHC became available to Haskell programs running in Web browser.<br /><br /><h2>7. Application coding paradigm</h2><br />At the moment DOM bindings were complete, it became possible to call DOM functions directly (in continuation-passing style). So Haskell program looked similarly to Javascript, and this way of coding seemed to require too much effort. It was the same old sequential programming. CPS might be replaced by monadic interface, but this did not change the matter. Some functional-reactive paradigm was necessary.<br /><br />The <a href="http://www.cs.chalmers.se/~hallgren/Thesis/main.html">Fudgets</a> library inspired some experiments. At the first attempt, the lowest-level layer of Stream Processors was successfully ported (tested on Firefox). Unfortunately, the Fudgets library itself showed some execution overhead, and finally when tested on MSIE, significant memory leaks.<br /><br />Later, CPS Monad was tried out (instead of plain CPS). Same situation: works on Firefox, leaks and freezes on MSIE.<br /><br />Plain CPS, though, never had such problems. So, after several months of experiments, it was finally decided to stay with plain CPS, and try to find a suitable solution to describe static Web page layout in mostly declarative way while leaving some degree of freedom for developers as to how to define the interactive (dynamic) part of an application.<br /><br />In addition, pseudo-threads (in fact, coroutines) and message boxes for inter-thread communication were included in the runtime making it easier to develop interactive, event-driven applications.<br /><br /><h2>8. Haskell web toolkit</h2><br />It turned out that a relatively thin layer of combinators on top of DOM may solve the problem of defining the static layout of Web page. Most of ideas were borrowed from the <a href="http://www.cse.ogi.edu/~andy/html/intro.htm">HTML</a> and its successor <a href="http://www.haskell.org/ghc/docs/latest/html/libraries/xhtml/Text-XHtml.html">XHTML</a> libraries (by Andy Gill, Bjorn Bringert).<br /><br />Static structure of a Web page is described as a hierarchy of widgets, some (containers, that correspond to HTML elements with opening and closing tags) may nest other widgets, and some are terminals (leaves, like text labels or images). So, developing such static layout does not differ much from writing HTML, only these constructs are not indeed static: powerful functions like <tt>map</tt> and <tt>fold</tt> may be used to operate over multiple widgets at once.<br /><br />Dynamics of the user interface can be added later, and it is almost completely orthogonal to static design. Special elements called activators expose the same interface as other widgets (that is, may be nested in containers), but their semantics includes runtime operations on their "parent" widgets such as handling user-induced events, on-the-fly style and content manipulation, or data exchange over XMLHTTP.<br /><br />The name picked, "Haskell Web Toolkit" unintentionally resembles "Google Web Toolkit", but indeed, they both serve the same purpose: bringing in some language not supported natively by Web browsers as a language to program for Web browsers. Weights and sizes of the two toolkits are of course incomparable ... yet ;)<br /><br /><h2>9. The first milestone</h2><br />As the Javascript backend's code matured, the idea appeared to set up a web service for everybody to submit their Haskell source code for compilation and instant loading into their Web browsers. Such service was implemented, based on <a href="http://incubator.apache.org/couchdb/">CouchDB</a> as intermediate storage. The service was quietly opened for public testing on March 4, 2008. Having such service available was considered reaching the first milestone in the Javascript backend project.<br /><br />So, what has been achieved since the first "Roman echo" demo was published:<br /><ul><br /><li>speed of generated Javascript was significantly increased, partly due to improvements in the code generator, partly thanks to the Yhc Core optimizer which was developed in the same time when Javascript backend</li><br /><li>a tool to automatically generate Haskell bindings from OMG IDL interface definitions was created. It is currently investigated whether Javascript documentation tools such as JsDoc could be used to generate OMG IDL definitions for existing Javascript libraries: this would make integration of them with Haskell much easier</li><br /><li>programming paradigm for Haskell Web applications was defined. The Haskell Web Toolkit library is not large at the moment, but it will definitely be extended with new pieces as usage of Haskell in Web programming gets wider</li><br /><li>a proof-of-concept live Yhc Web service was launched, so everybody interested may try Haskell in their browsers without installation of any software</li></ul><br /><br /><h2>10. Acknowledgements</h2><br />Thanks to the Yhc Team for helping understand some internals of the compiler. Special thanks to Neil Mitchell who developed the Yhc Core format and the Core tools library. This helped a lot to improve the quality of Javascript generated by the backend. Some people contacted me privately and suggested some ideas or identified bugs. I am not putting names here, but feel free to identify yourselves in the comments.<br /><br />And most of all: interested developers are welcome! The more people contribute in this project, the more chance Haskell has to become wider used in Web applications whose importance grows over time.<br /><br /><h2>Links</h2><br /><ul><li><a href="http://www.haskell.org/haskellwiki/Haskell_in_web_browser">Haskell in Web Browser, a Tutorial</a></li><br /><li><a href="http://www.haskell.org/haskellwiki/Yhc_web_service">Yhc Web Service, Users Guide</a></li><br /><li><a href="http://www.golubovsky.org:5984/static/yhcws/MainGUI.html">Yhc Web Service live</a> [defunct as of March 2009]</li><br /><li><a href="http://www.golubovsky.org:5984/_utils/yhcws/index.html">API Documentation (Haddock)</a> [defunct as of March 2009]</li><br /></ul>Dimitry Golubovskyhttp://www.blogger.com/profile/10316900314693187190noreply@blogger.com3tag:blogger.com,1999:blog-18841470.post-1170694249293700222007-02-05T16:38:00.000+00:002007-02-05T16:50:49.303+00:00100 bugs!Three days ago Thorkil Naur filed <a href="http://code.google.com/p/yhc/issues/detail?id=100&can=2&q=">bug #100</a> for Yhc. Using a bug tracker was a good step for Yhc - and has helped us remember, track down and fix loads of bugs. We currently have 101 bugs, 62 of which remain open. Some bug statistics:<br /><ul><li>7 <a href="http://code.google.com/p/yhc/issues/list?q=label:beginner">beginner bugs</a> - these are bugs which don't require much Haskell knowledge, or in depth compiler knowledge. If you are just getting started, all of these should be within reach.</li><li>15 defects which are either High or Medium importance - at least one of which got fixed this morning!</li><li>30 <a href="http://code.google.com/p/yhc/issues/list?can=2&q=label%3AType-Enhancement&colspec=ID+Type+Status+Priority+Milestone+Owner+Summary">enhancement requests</a> - often relating to cleaning up the internals of Yhc.</li><li>17 bugs owned by me (2 by Andrew, 4 by Dimitry, 2 by Tom, 2 by Mike) - 25 waiting for you to own them :)</li></ul>Well done to everyone who has contributed to Yhc - and hopefully we'll be able to reduce the bug count to 0 at some point!Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1166660487156575712006-12-21T00:19:00.000+00:002006-12-21T00:21:27.170+00:001000 patches!The Yhc project hit 1000 patches sometime today. Congratulations on Dimitry for being the 1000th patcher, and for everyone who's made a commit.<br /><br />This does of course mean that doing a non --partial darcs get is going to take<span style="font-style: italic;"> forever</span>...Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1166056993820772302006-12-14T00:36:00.000+00:002006-12-14T00:43:13.903+00:00Yhc.Core.HtmlThe Yhc Core stuff has been coming on loads recently, being used in more and more projects, and gaining lots of cool features - even a strictness analyser! Check out the Yhc documentation here: <a href="http://www.cs.york.ac.uk/fp/yhc/snapshot/docs/">http://www.cs.york.ac.uk/fp/yhc/snapshot/docs/</a> . A lot of this work is focused towards performance enhancements, so I'll give more details on that once we have a headline number to brag about :)<br /><br />One cool thing I've been working is Yhc.Core.Html (not yet in the documentation index, but it should be tomorrow). If you load up Hugs and take your Yhc.Core file:<br /><br />> loadCore "Roman.ycr" >>= writeFile "Roman.html" . coreHtml<br /><br />That will generate a file called Roman.html, which is a hyperlinked, colour-coded viewer for your code. A <a href="http://www.cs.york.ac.uk/fp/yhc/Roman.html">sample is here</a>. The links which are internal to this file are hyperlinked, and it also hilights the uses as well. Click on an identifier to see what it does.<br /><br />Hopefully this should be of use to people who spend most of their day pouring over the details in a Yhc.Core document - i.e. me!Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1160953620422203762006-10-16T00:02:00.000+01:002006-10-16T00:07:00.436+01:00Yhc.Core documentation!Since a few people now have an interest in Yhc.Core I thought I'd actually document it with a little tutorial style introduction.<br /><br />There are also some little examples, including one to count the number of times you use the literal 42 in your code.<br /><br /><a href="http://haskell.org/haskellwiki/Yhc/API/Core#Some_little_samples">http://haskell.org/haskellwiki/Yhc/API/Core#Some_little_samples</a><br /><br />We've also been busy getting installing set up, you can now do "scons install" on Linux, download nice snapshots on Windows, and there is progress towards Gentoo eBuilds and Debian .deb's.Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com2tag:blogger.com,1999:blog-18841470.post-1160325020901879862006-10-08T17:12:00.000+01:002006-10-08T17:30:21.073+01:00Beginner BugsWant to start helping out on a big Haskell project, but frustrated that there is no easy way into a project? The Yhc team has the answer, help with Yhc :)<br /><br />There are basically 3 steps to get into full swing as a proper Yhc developer, and everyone is welcome!<br /><br />1) Download Yhc from darcs, and compile. <a href="http://www.haskell.org/haskellwiki/Yhc">http://www.haskell.org/haskellwiki/Yhc</a><br /><ul><li>If it goes wrong, email us (yhc -at- haskell.org)</li><li>If the documentation is unclear, fix it or email us</li><li>If something takes some hard work, tell us what, so we can fix it<br /></li></ul>2) Either pick your own task you feel like, or pick one of our <a href="http://code.google.com/p/yhc/issues/list?q=Beginner">beginner bugs</a> from the bug tracker. All the beginner bugs are designed to be able to be done incrementally, i.e. even a small amount of progress can be used. They shouldn't require much deep knowledge of the code base, some require a bit of Haskell, some require a bit more, some require none. There should be something suitable for everyone on that list.<br /><br />3) Either pick your own task, or pick one of our <a href="http://code.google.com/p/yhc/issues/list">normal bugs</a> - they vary greatly in difficulty, scope etc. By this time you are a fully fledged Yhc developer :)<br /><br />We are friendly people, so if at any point you need a bit of advice or help, or something seems to be a bit harder than it could be, just drop us an email - yhc -at- haskell.orgNeil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1158777271614169972006-09-20T19:23:00.000+01:002006-09-20T19:34:31.633+01:00Yhc.Core API availableYhc.Core is now a publically available and somewhat supported API! There are no docs on it, but check out the Yhc repo and see src/compiler98/Yhc/Core. Just add "import Yhc.Core" at the top and you can do all cool sorts of things.<br /><br />Step 1 is to generate Core files, with the .ycr extension. If you are compiling Yhc adding scons core=1 to the command line, and all the libraries will have .ycr files generated along with the .hi and .hbc files. If you are compiling an individual file, then add the yhc -corep flag, and it will generate a .ycr file.<br /><br />The .ycr files are stored in Binary, but thats OK, because if you want to view one, its literally as simple as:<br /><br />import Yhc.Core<br />showFile x :: FilePath -> String<br />showFile x = loadCore x >>= return . show<br /><br />(There will be a flag/program/command to view a .ycr file at some point - its not hard to do, but it is hard to figure out where the flag should go and what it should be called!)<br /><br />Now once you've got Core, what can you do? Take a look at the data structure in the <a href="http://www.cs.york.ac.uk/fp/darcs/yhc-devel/src/compiler98/Yhc/Core/Type.hs">Yhc.Core.Type</a> file, to see what it contains. There are already two programs that depend on Yhc.Core, <a href="http://www-users.cs.york.ac.uk/%7Endm/projects/catch.php">Catch</a> and <a href="http://www-users.cs.york.ac.uk/%7Endm/projects/drhaskell.php">Dr Haskell</a>. In particular if you take a look at the <a href="http://www.cs.york.ac.uk/fp/darcs/drhaskell/Main.hs">main file</a> of Dr Haskell, you can see some trivial ways to play with Core.<br /><br />Hopefully Yhc.Core will be useful for many projects - if it interests you please drop an email to yhc -AT- haskell.org, saying what you want to do etc. (Of course, you don't have to, but we're curious folk!)Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1158460460849079262006-09-17T03:25:00.000+01:002006-09-17T03:34:20.860+01:00Nofib suite getting there!Some good news to report, slowly but surely more of the nofib benchmark suite is running successfully - we now have 24 tests in the nofib benchmark going through nicely, and added to the <a href="http://www.indiegigs.co.uk:8010/">buildbot</a>.<br /><br />The <a href="http://code.google.com/p/yhc/issues/list">issue tracker</a> is starting to see more use, getting more of the thoughts that were marked down as "oh, we should do that" and converting them into concrete bugs that one day someone might solve. Its also useful for tracking down where things are going.<br /><br />We can now output core for all the libraries Yhc builds with - simply pass core=1 to Scons and it will do the hard work for you. Not massively helpful, unless you're me, but might be of general use one day!<br /><br />We also have a few more features merged in - no need to give the -unlit flag for .lhs files, and the OPTIONS_YHC pragma is supported in a similar manner to the <a href="http://www.haskell.org/ghc/docs/6.4.2/html/users_guide/using-ghc.html#source-file-options">OPTIONS_GHC</a> pragma.<br /><br />Yhc is slowly getting more useful, once the haskell.org base libraries are useable with Yhc, it will be good enough to start eating our own dogfood!Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1155482914262726532006-08-13T16:18:00.000+01:002006-08-13T16:28:34.273+01:00Buildbot!GHC has nightly builds, now Yhc has something even better, buildbot!<br />http://www.indiegigs.co.uk:8010/ - and for instructions http://www.haskell.org/haskellwiki/Yhc/Buildbot (email yhc -AT- haskell.org to register)<br /><br />Every time a change it checked into the Yhc repo, builds are triggered, and the test results are calculated. This should ensure that breaking the build is a 10 minute affair, and it also means we can remotely debug compile/runtime errors on other people's machines by trail and error without their intevention - which has already meant that we got Mac PPC up and running without access to the machine. Something that would have taken much longer if we had to wait for email exchanges between every change.<br /><br />So who has a wacky architecture, and wants to be a buildbot? Its entirely automated, you can nice is as much as you want, your machine doesn't have to be on all the time, and it means that you'll always have a working Yhc available for you :)Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com1tag:blogger.com,1999:blog-18841470.post-1153749182929339182006-07-24T13:59:00.000+01:002006-07-24T14:53:02.986+01:00Yhc Core (v2)I have tweaked the Yhc -core output so its much more like normal Haskell, still not 100% there, but getting closer all the time :)<br /><br />I want it to be possible to do:<br />yhc -core Sample.hs 2> Sample2.hs<br />hugs Sample2.hs<br /><br />At the moment, that sometimes works. The only problem I know about is dictionaries for typeclasses - and thats all to do with their names. I'm still figuring out what the answer to that should be.<br /><br />Anyway, taking the pam daeh example thats been used before:<br /><br /><pre><br />daeh v220 =<br /> case v220 of<br /> (:) v221 v222 -><br /> v221<br /> _ -><br /> Prelude.error Sample._LAMBDA228<br /><br />_LAMBDA228 =<br /> "Sample: Pattern match failure in function at 9:1-9:15."<br /><br />pam v223 v224 =<br /> case v224 of<br /> [] -><br /> []<br /> (:) v225 v226 -><br /> (:) (v223 v225) (Sample.pam v223 v226)<br /><br />test v227 =<br /> Sample.pam Sample.daeh v227<br /></pre><br /><br /><br />And now the cool bit:<br /><br />hugs Sample2.hs<br /><br />And it works!<br /><br />Hopefully this will be of use to people who want to work with reduced Haskell, since they can use existing Haskell parsers and type checkers to do their work. If you are interested in this, drop an email to the yhc mailing list (yhc -AT- haskell.org).Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1149091866900062092006-05-31T17:10:00.000+01:002006-05-31T17:11:06.923+01:00Summer of Code (2)<div class="extern_app"> <div> <strong>Implement a better type checker for yhc</strong> </div> <div> <small>by Leon P Smith, mentored by Malcolm Wallace<br /><br />We have a change in the student, so good luck to Leon on the type checker!<br /></small> </div> </div>Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1148661585413374812006-05-26T17:38:00.000+01:002006-05-26T17:39:45.423+01:00Summer of Code<a href="http://code.google.com/soc/haskell/about.html">http://code.google.com/soc/haskell/about.html</a><br /><br /><div class="extern_app"> <div> <strong>Implement a better type checker for Yhc</strong> </div> <div> <small>by Mathieu Boespflug, mentored by Malcolm Wallace</small> </div> </div><br />Here's hoping for a better type checker!Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1142263249368025362006-03-13T15:18:00.000+00:002006-03-13T15:20:49.386+00:00Patch StatisticsEver wondered how often people commit patches to Yhc?<br /><br />Now, a nice graphical chart can show you:<br /><a href="http://www-users.cs.york.ac.uk/~ndm/temp/yhc-patches.html">http://www-users.cs.york.ac.uk/~ndm/temp/yhc-patches.html</a><br /><br />Note that this page won't be around for ever, but I hope that once we have haddock documentation and regression tests running every night, slipping this in as well won't be too hard, or perhaps it might get into darcs-web.<br /><br />The graph goes from Nov 2005 to present, and the current number of patches is about 340.<br /><br />The tool for generating this graph is in the repo, under misc/DarcsGraph.hs - just run "darcs changes" to a file, and give that file to DarcsGraph.Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com4tag:blogger.com,1999:blog-18841470.post-1141320881463289962006-03-02T17:32:00.000+00:002006-03-02T17:34:41.476+00:00Pyhi: A python interpreter<a href="http://www-users.cs.york.ac.uk/~aw/">Andrew</a> says:<br /><br />I've just committed an entirely new Yhc interpreter, written in Python. Pyhi <br />(as it is known) currently only supports a limited set of the bytecodes <br />(enough to run Hello World) and is approximately 100 times slower than the C <br />version.<br /><br />It uses many of the built in Python features, such as garbage collection, to <br />attempt to gain some speed and yet is still hopefully written in a fairly <br />understandable, Pythonic fashion.<br /><br />Bytecodes are represented as functions which are called using a 'trampoline' <br />function in order to decouple the Haskell and Python call stacks.<br /><br />This most definitely a work in progress so don't expect many programs will run <br />just yet :-)Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1141050640016900452006-02-27T14:27:00.000+00:002006-02-27T14:30:40.040+00:00CpphsYhc now has full and nice support for C Preprocessor directives. If you would like to compile with them simply include the flag -cpp, and it will run an internal preprocessor (no extra runtime dependancies).<br /><br />The cpp support is provided by <a href="http://www.cs.york.ac.uk/fp/cpphs/">Cpphs</a> in its use as a library. For the moment, the flag defined is __YHC__. The makefile on both Windows and Unix has changed substantially - both automatically detect if you are missing cpphs, and darcs get it for you if required.<br /><br />Hopefully this should make stage 2 easier - compiling the heirarchical libaries!Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1137784769148789812006-01-20T19:13:00.000+00:002006-01-20T19:19:29.176+00:00Yhc vs YHCWe have now named most things, which is a good step forward (Yhc, Yhi, Yhe, Gyhe, Yho ...)<br /><br />One final thing we need to name is Yhc itself. If you look at the previous posts you'll see I tend to name it Yhc, and Bob tends to name it YHC. I personally prefer Yhc, because it looks quite a lot nicer, all the demo logos that have been done have had "hc" in lower case, and YHC seems a lot more violent, while Yhc is more friendly.<br /><br />Thoughts and opinions?Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com2tag:blogger.com,1999:blog-18841470.post-1137285258703420352006-01-15T00:29:00.000+00:002006-01-16T03:03:14.823+00:00Wiki v2I have started moving some bits of the <a href="http://haskell.org/hawiki/Yhc">old wiki</a> to the <a href="http://haskell.org/haskellwiki/Yhc">new wiki</a>, there is quite a lot of content, so if anyone wants to help :) I'm also thinking the <a href="http://www-users.cs.york.ac.uk/%7Endm/yhc/">Yhc website</a> should just be the wiki, no static pages at all - so all the static content probably needs moving as well.<br /><br />Hopefully once all this is done we'll have the best documented compiler.<br /><br /><span style="font-weight: bold;">Update: </span>all done, there should be nothing on the old wiki which isn't on the new one - I also reorganised quite a bit at the same time.Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1136939669151819642006-01-11T00:32:00.000+00:002006-01-11T00:34:29.166+00:00Lines of codeFor anyone who's interested, currently the Yhc compiler is 23578 lines of Haskell code. This is just the compiler, not the runtime etc.Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1136293395127415632006-01-03T12:54:00.000+00:002006-01-03T13:03:15.136+00:00User InterfaceThe Yhc compiler takes most of its Haskell-compiling code from nhc98, and most of its backend/interpretter has been rewritten by Tom. The one bit that I think needs most attention is the user interface. Specifically, I see the following problems in the user interface (where user is a programmer, and interface is how they use yhc):<br /><ul><li> YHC_BASE_PATH - this trips pretty much everyone up</li><li>Path handling, where .hbc files go etc</li><li>Doesn't list which file its compiling.</li><li>Error messages are usually pretty bad, compiling many files at a time has made this worse, since now you don't even know the file the errors in</li></ul>YHC_BASE_PATH is scheduled for elimination. Listing compiled files is trivial. Error messages are being slowly replaced, but the lack of global state in Haskell is making this quite a bit harder.<br /><br />The one thing where there is less concensus as to what the desired behaviour should be is path handling - i.e. deciding where to put a file, and where to get it from. I have started a wiki page with my thoughts on this, which is <a href="http://www.haskell.org/hawiki/Yhc/PathHandling">here</a>. Please add your thoughts, hopefully we'll come to a nice concensus, and have a nicely implemented user interface before long.Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1136172657586054302006-01-02T01:59:00.000+00:002006-01-02T03:32:09.966+00:00HsNub - updateI just finished HsNub, and while it works basically, its not all that I was hoping for. The basic problem is that lots of functions which are entirely different have the same bytecode! For example, selecting the first element out of a data structure is given as lots of different names, often field selectors. To be useful, it would need to incorporate some knowledge of the types as well.<br /><br />Just to show an example of it running, when trying it on my regular expression library, tweaked to add two functions func1 and func2 that are just like id, I get the result:<br /><br />{Data.Char-cont, YHC.Internal-_id, Prelude-id, Prelude-Prelude.Enum.Prelude.Char.fromEnum, Prelude-Prelude.Enum.Prelude.Char.toEnum, Prelude-Prelude.Integral.Prelude.Integer.toInteger, Prelude-Prelude.Num.Prelude.Integer.fromInteger, RegExpData-func2, RegExpData-func1}<br />(plus plenty of other equal functions)<br /><br />So it works, but its just a bit too verbose, and your program must type, whereas the bytecode doesn't. Its also really slow, but I'm sure that could be fixed with a bit of effort.<br /><br />One thing I did find is that a massive amount of the standard libraries are the same. Some statistics about the functions in the prelude:<br /><br />1826 functions<br />701 distinct bytecode sequences (38%)<br />45025 bytecodes, 27584 of which are required (61%)<br />1363 distinct constant tables (75%)<br />4732 constant entires, 4512 of which are required (95%)<br /><br />Maybe a better bytecode format would store each function as a pointer to a bytecode chunk, and a pointer to a constant table - that way different ones could be mixed and matched. I don't expect the standard compiler to do this (way too expensive, harms debugging), but its useful to be able to do as a standalone optimiser, and nicely cuts away at the memory required.Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com1tag:blogger.com,1999:blog-18841470.post-1135608860652342322005-12-26T14:46:00.000+00:002005-12-26T14:54:20.663+00:00Crazy IdeasHaving a small, clean, simple, production strength Haskell compiler lets a lot of people come up with Crazy ideas that just weren't feasible before. (alright, so Yhc certainly isn't production strength yet, but it could be without too much work...)<br /><br />Some of the crazy ideas are listed at <a href="http://www.haskell.org/hawiki/Yhc/Ideas">on the wiki</a>, some others have been sent <a href="http://www.haskell.org//pipermail/yhc/2005-December/000039.html">out in emails</a>. Quite a lot aren't crazy at all, but would be really useful! Just a quick summary of some side projects/tools that are being developed, or have been thought of:<br /><ul><li>Haskell HBC -> Java bytecode</li><li>Javascript runtime for HBC files</li><li>Python runtime for HBC files</li><li>Playstation port (entirely speculation so far)</li></ul>I also came up with a new crazy idea for Yhc, HsNub. Have you ever written a function, and later found out that it did the same as a function in the prelude? Take the Yhc code base, I recently eliminated the function <span style="font-style: italic;">second</span>, which is equivalent to the Prelude function <span style="font-style: italic;">const</span>. It would have been nice if something spotted this and gave out a warning. With a portable bytecode, this isn't that much work - two functions can be seen as equivalent if they are identical at the bytecode level. I intend to write this tool once I have a working bytecode library - I don't expect it to take more than an hour at most - thats the nice thing about having a relatively simple Haskell implementation!Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com3tag:blogger.com,1999:blog-18841470.post-1134334582895976512005-12-11T20:43:00.000+00:002005-12-11T20:56:22.910+00:00Yhc CoreSome people might have seen or heard of GHC's Core language, there is a detailed account of it <a href="http://www.cse.unsw.edu.au/%7Echak/haskell/ghc/comm/the-beast/desugar.html">here</a>. As part of my <a href="http://www-users.cs.york.ac.uk/%7Endm/projects/catch.php">PhD</a> I need to convert from full Haskell to a reduced Haskell-like language. How should I go about doing this? Well the obvious thing is to use a Haskell compiler to translate to a reduced language, then tweak this reduced language to my personal taste. A few months ago when I started this process, the options available to me were:<br /><ul><li>GHC - a massive monster, but with an emerging API, and somewhat complex Core</li><li>Hugs - C, yuk!</li><li>nhc98 - doesn't work on Windows</li></ul>With a choice like that, I went for GHC and started doing some work. It turns out the GHC Core that I was getting had let's, so learning let floating seemed necessary. Also all the symbols are nastily mangled - GHCziBase.ZMZN is the Core way of saying []. Also there is no natural correspondance between the original and the new names, and multi-module stuff looked like it was going to be a problem as well - HsAllInOne seemed to do the business, but thats yet another layer of mangling on top of that, which makes it look revolting.<br /><br />Well that was 3 months ago I think, and since then an extra compiler has emerged, Yhc! So I thought I'd fire off Yhc and see what i got. Amazingly, I just compiled with -lift and suddenly everything looked beautifully close to what i wanted. The syntax was horrible, but it looked very nice! From this initial observation, I've hacked around a bit to try and define an easily parseable Core language for Yhc, which external tools (i.e. mine) can work with. It's a bit different from GHC's, compare and contast:<br /><blockquote> {from1 :: GHCziBase.Int -> GHCziBase.ZMZN GHCziBase.Int =<br /> \ (n::GHCziBase.Int) -><br /> GHCziBase.ZC @ GHCziBase.Int n<br /> (from1<br /> (%case (GHCziBase.Int) zddNum<br /> %of (tpl::GHCziNum.ZCTNum GHCziBase.Int)<br /> {GHCziNum.ZCDNum<br /> (tpl1::GHCziBase.ZCTEq GHCziBase.Int)<br /> (tpl2::GHCziShow.ZCTShow GHCziBase.Int)<br /> (tpl3::GHCziBase.Int -> GHCziBase.Int -> GHCziBase.Int)<br /> (tpl4::GHCziBase.Int -> GHCziBase.Int -> GHCziBase.Int)<br /> (tpl5::GHCziBase.Int -> GHCziBase.Int -> GHCziBase.Int)<br /> (tpl6::GHCziBase.Int -> GHCziBase.Int)<br /> (tpl7::GHCziBase.Int -> GHCziBase.Int)<br /> (tpl8::GHCziBase.Int -> GHCziBase.Int)<br /> (tpl9::GHCziNum.Integer -> GHCziBase.Int) -><br /> tpl3 n lit}))};<br /></blockquote>vs:<br /><blockquote>Main.from v379 =<br /> (Prelude.: v379 (Main.from (Prelude.Prelude.Num.Prelude.Int.+ v379 1)))<br /></blockquote>A couple of differences to note. Yhc has no types, GHC has lots. Yhc resembles Haskell, GHC doesn't. The names are much clearer in Yhc. Yhc has a nice property that if you just cat all the modules together, it works perfectly. Another difference not shown here is that Yhc knows exactly where Main.from is defined, and can even track down exactly where that constant 1 comes from. It doesn't display this information, but it could do.<br /><br />It finally looks like I've got a way to make Yhc part of my Phd!<br /><br />[Note: all this code isn't in the build, its only in my copy, but it didn't take more than about 2 hours. Hopefully it can get into the main Yhc sometime soon]Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0tag:blogger.com,1999:blog-18841470.post-1134054952222910712005-12-08T15:09:00.000+00:002005-12-08T15:15:52.233+00:00GHC SurveyThe GHC team did a huge survey of Haskell users, its online <a href="http://haskell.org/ghc/survey2005-summary.html">here</a><br /><br />Its quite interesting to go down the points, and see which things that Haskell users want, but can't find in GHC, and Yhc might be able to provide in the future.<br /><br /><ul><li>GHC is too slow, Yhc will be faster</li><li>Performance and size of generated code - we win on size, loose massively on performance</li><li>Most requested feature is a debugger, I think we can win there :)</li><li>Portability and difficulty of compiling GHC, we can do that easily</li><li>API and feature changes, we loose massively</li><li>Quality of error messages, lets not go into that, but we can hopefully do better</li><li>Better documentation, well library docs we steal from GHC (draw), compiler docs are a wiki so hopefully that will be better quicker</li><li>GHC is too complex, well yhc is certainly a lot simpler</li></ul>Hopefully we can learn from the work of others to give people what they want!Neil Mitchellhttp://www.blogger.com/profile/13084722756124486154noreply@blogger.com0