From dbc@rpath.com Wed Sep 20 17:42:54 2006 Received: from lambchop.rdu.rpath.com (rdu-nat.rpath.com [66.192.95.194]) by lists.rpath.com (8.13.6/8.13.6) with ESMTP id k8KLgsqu007739 for ; Wed, 20 Sep 2006 17:42:54 -0400 Received: from [172.16.58.67] (wallace.rdu.rpath.com [172.16.58.67]) by lambchop.rdu.rpath.com (8.13.6/8.13.4) with ESMTP id k8KLfINR021483 for ; Wed, 20 Sep 2006 17:41:18 -0400 Message-ID: <4511B568.1010603@rpath.com> Date: Wed, 20 Sep 2006 17:40:56 -0400 From: David Christian User-Agent: Thunderbird 1.5 (X11/20060113) MIME-Version: 1.0 To: rmake-discuss@lists.rpath.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Wed, 20 Sep 2006 18:22:43 -0400 Subject: [Rmake-discuss] Test message X-BeenThere: rmake-discuss@lists.rpath.com X-Mailman-Version: 2.1.6 Precedence: list List-Id: Discussion regarding the rMake build tool List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Sep 2006 21:42:54 -0000 test From dbc@rpath.com Fri Sep 29 13:10:46 2006 Received: from lambchop.rdu.rpath.com (rdu-nat.rpath.com [66.192.95.194]) by lists.rpath.com (8.13.6/8.13.6) with ESMTP id k8THAj3v007547 for ; Fri, 29 Sep 2006 13:10:45 -0400 Received: from [172.16.65.13] (dbc.rdu-vpn.rpath.com [172.16.65.13]) by lambchop.rdu.rpath.com (8.13.6/8.13.4) with ESMTP id k8TH95sX023481 for ; Fri, 29 Sep 2006 13:09:05 -0400 Message-ID: <451D5320.7050301@rpath.com> Date: Fri, 29 Sep 2006 13:08:48 -0400 From: David Christian User-Agent: Thunderbird 1.5 (X11/20060113) MIME-Version: 1.0 To: rmake-discuss@lists.rpath.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Rmake-discuss] Bug filing policy X-BeenThere: rmake-discuss@lists.rpath.com X-Mailman-Version: 2.1.6 Precedence: list List-Id: Discussion regarding the rMake build tool List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2006 17:10:46 -0000 Hi folks, sorry to use this as an announcement list, but I don't have an announcement list so this will have to do. After having just spent quite a bit of time trying to reproduce a bug that was underspecified, I'd like to ask that _all_ rMake bugs about build behavior be filed with: 1. output of "rmake config". 2. The full version of the trove you're building. Generally that, and a description of what is happening, is enough for me to reproduce. Without that information, however, tracking down exactly what is going on can be a nightmare/impossible. Thanks for your help in cutting down the number of iterations that rmake bugs have to go through before they can be fixed. New rMake release coming soon. Early next week we're talking more in house about how to support larger rMake projects, when we've got a draft spec written up I'll send it here. Dave From dbc@rpath.com Wed Oct 25 17:21:59 2006 Received: from lambchop.rdu.rpath.com (rdu-nat.rpath.com [66.192.95.194]) by lists.rpath.com (8.13.6/8.13.6) with ESMTP id k9PLLskL023600 for ; Wed, 25 Oct 2006 17:21:54 -0400 Received: from [172.16.58.67] (wallace.rdu.rpath.com [172.16.58.67]) by lambchop.rdu.rpath.com (8.13.6/8.13.4) with ESMTP id k9PLK1TG025827 for ; Wed, 25 Oct 2006 17:20:01 -0400 Message-ID: <453FD508.6010300@rpath.com> Date: Wed, 25 Oct 2006 17:20:08 -0400 From: David Christian User-Agent: Thunderbird 1.5 (X11/20060113) MIME-Version: 1.0 To: rmake-discuss@lists.rpath.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Rmake-discuss] Projects - a new way of using rMake X-BeenThere: rmake-discuss@lists.rpath.com X-Mailman-Version: 2.1.6 Precedence: list List-Id: Discussion regarding the rMake build tool List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Oct 2006 21:22:01 -0000 Here's a rough idea of something we are looking at doing for rMake in order to allow users to perform some more complicated actions with rMake. We call them projects. Here's the basic way it would work: You create a new project using rmake newproject. "rmake project foo" will set the project to use in the current directory, or you can set the "project" config value in your context to let rmake know what project you're working on. Then add a package/set of packages to your project, using "rmake add" commands that match what you would use for "rmake build" now. Rmake will track exactly what command you used. So, for example, if you use "rmake add foo=1.5", rmake will remember that you said to search for version 1.5 and find the latest version that matches. Then run "rmake build" to build the entire project. Alternately, you can run "rmake build blah" to build "blah" in the context of your project. When you rerun "rmake build", rMake will search for updates to the packages you add to their buildreqs and rebuild them if anything's changed. So, if the upstream distro changes openssl, for example, you would rebuild your packages with the new openssl. (The exact behavior of this will be configurable, so that if you want to avoid rebuilding due to a particular change, you can do that). When you run "rmake commit", rMake will commit binaries and sources like it does now. The basic goal of this first step is to allow you to manage a set of packages for your distribution and repeatedly rebuild the distro. Longer term, we're looking at adding the concept of "conditional rebuilds", so that you can build a package (say, something that you derive from upstream) if and only if something else you build requires it. Basically, if you use rPath's packages, and you update one of their buildreqs, we'd like rMake to be able to automatically recognize that you are going to have to rebuild those packages. But that's longer term. We're looking for extra requirements or feature requests for this first scope, whether you think you'd use it, and so on. From dbc@rpath.com Wed Oct 25 17:53:46 2006 Received: from lambchop.rdu.rpath.com (rdu-nat.rpath.com [66.192.95.194]) by lists.rpath.com (8.13.6/8.13.6) with ESMTP id k9PLrjV1023720 for ; Wed, 25 Oct 2006 17:53:45 -0400 Received: from [172.16.58.67] (wallace.rdu.rpath.com [172.16.58.67]) by lambchop.rdu.rpath.com (8.13.6/8.13.4) with ESMTP id k9PLpt0N009091 for ; Wed, 25 Oct 2006 17:51:55 -0400 Message-ID: <453FDC81.9020607@rpath.com> Date: Wed, 25 Oct 2006 17:52:01 -0400 From: David Christian User-Agent: Thunderbird 1.5 (X11/20060113) MIME-Version: 1.0 To: rmake-discuss@lists.rpath.com Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: [Rmake-discuss] Road Map through the rest of the year X-BeenThere: rmake-discuss@lists.rpath.com X-Mailman-Version: 2.1.6 Precedence: list List-Id: Discussion regarding the rMake build tool List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Oct 2006 21:53:46 -0000 Hey guys, just to let you know what I'm planning on doing: This month: I'd like to get something like projects out the door. I'm also planning on implementing as much caching as I can short of chroot caching - since chroot caching is likely to require a bit more time than I have to spare. Next month: I'd like to get chroot caching done as well as support for multiple build nodes, although I don't know if the code for multiple build nodes will be available to the general public (it may be for our customers only). January: I'll be working on the steps needed to make rMake support a complete distro rebuild from bootstrapping to completion. I expect throughout this time I'll be working on improving whatever Projects turn into - that will probably take up a lot of my time. If we add developers, we'll probably spend the extra time working on features related to making them more useful. Please let me know if there's something missing from this road map that you desperately need! Dave From jtate@rpath.com Thu Oct 26 08:33:06 2006 Received: from lambchop.rdu.rpath.com (rdu-nat.rpath.com [66.192.95.194]) by lists.rpath.com (8.13.6/8.13.6) with ESMTP id k9QCX63g028882 for ; Thu, 26 Oct 2006 08:33:06 -0400 Received: from [172.16.58.74] (router.rdu.rpath.com [172.16.58.1]) (authenticated bits=0) by lambchop.rdu.rpath.com (8.13.6/8.13.4) with ESMTP id k9QCVEAn013369 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Thu, 26 Oct 2006 08:31:16 -0400 Message-ID: <4540AB01.8000804@rpath.com> Date: Thu, 26 Oct 2006 08:33:05 -0400 From: Joseph Tate User-Agent: Thunderbird 1.5.0.5 (X11/20060727) MIME-Version: 1.0 To: rmake-discuss@lists.rpath.com References: <453FD508.6010300@rpath.com> In-Reply-To: <453FD508.6010300@rpath.com> X-Enigmail-Version: 0.94.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: [Rmake-discuss] Projects - a new way of using rMake X-BeenThere: rmake-discuss@lists.rpath.com X-Mailman-Version: 2.1.6 Precedence: list List-Id: Discussion regarding the rMake build tool List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Oct 2006 12:33:06 -0000 David Christian wrote: > Longer term, we're looking at adding the concept of "conditional > rebuilds", so that you can build a package (say, something that you > derive from upstream) if and only if something else you build requires > it. Basically, if you use rPath's packages, and you update one of their > buildreqs, we'd like rMake to be able to automatically recognize that > you are going to have to rebuild those packages. But that's longer term. Is that even necessary? It may be good practice, but unless something else has changed (different glibc or compiler), do you really need a rebuild if libjpeg has bumped without changing ABI? That's a lot of packages to rebuild. I think that you leave it to the developer to say whether this particular change merits rebuilding everything that depends on it. -- Joseph Tate Software Engineer rPath Inc http://www.rpath.com/rbuilder/ +1-919-851-3984 x2106 From smithj@rpath.com Thu Oct 26 09:58:24 2006 Received: from lambchop.rdu.rpath.com (rdu-nat.rpath.com [66.192.95.194]) by lists.rpath.com (8.13.6/8.13.6) with ESMTP id k9QDwO4x029393 for ; Thu, 26 Oct 2006 09:58:24 -0400 Received: from [172.16.58.85] (router.rdu.rpath.com [172.16.58.1]) (authenticated bits=0) by lambchop.rdu.rpath.com (8.13.6/8.13.4) with ESMTP id k9QDuVxH017483 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 26 Oct 2006 09:56:32 -0400 Message-ID: <4540BE8F.3090309@rpath.com> Date: Thu, 26 Oct 2006 09:56:31 -0400 From: Jonathan Smith User-Agent: Thunderbird 1.5.0.7 (X11/20060914) MIME-Version: 1.0 To: Joseph Tate References: <453FD508.6010300@rpath.com> <4540AB01.8000804@rpath.com> In-Reply-To: <4540AB01.8000804@rpath.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: rmake-discuss@lists.rpath.com Subject: Re: [Rmake-discuss] Projects - a new way of using rMake X-BeenThere: rmake-discuss@lists.rpath.com X-Mailman-Version: 2.1.6 Precedence: list List-Id: Discussion regarding the rMake build tool List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Oct 2006 13:58:25 -0000 Joseph Tate wrote: > David Christian wrote: >> Longer term, we're looking at adding the concept of "conditional >> rebuilds", so that you can build a package (say, something that you >> derive from upstream) if and only if something else you build requires >> it. Basically, if you use rPath's packages, and you update one of their >> buildreqs, we'd like rMake to be able to automatically recognize that >> you are going to have to rebuild those packages. But that's longer term. > > Is that even necessary? It may be good practice, but unless something > else has changed (different glibc or compiler), do you really need a > rebuild if libjpeg has bumped without changing ABI? That's a lot of > packages to rebuild. > > I think that you leave it to the developer to say whether this > particular change merits rebuilding everything that depends on it. > Yes, you do need to rebuild. If, say, libjpeg:devel is in the build reqs, it is impossible for rMake to know what part of libjpeg is used - could be the dynamic libs (which are actually part of libjpeg:lib), could be the static libs (which *would* necessitate a rebuild) or could even be some program who's behavior has changed. -smithj From johnsonm@rpath.com Thu Oct 26 10:21:46 2006 Received: from lambchop.rdu.rpath.com (rdu-nat.rpath.com [66.192.95.194]) by lists.rpath.com (8.13.6/8.13.6) with ESMTP id k9QELkd1029483 for ; Thu, 26 Oct 2006 10:21:46 -0400 Received: from lambchop.rdu.rpath.com (localhost.localdomain [127.0.0.1]) by lambchop.rdu.rpath.com (8.13.6/8.13.4) with ESMTP id k9QEJtGo027334 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 26 Oct 2006 10:19:55 -0400 Received: (from johnsonm@localhost) by lambchop.rdu.rpath.com (8.13.6/8.13.4/Submit) id k9QEJtmm027330; Thu, 26 Oct 2006 10:19:55 -0400 Date: Thu, 26 Oct 2006 10:19:55 -0400 From: "Michael K. Johnson" To: Jonathan Smith Message-ID: <20061026141954.GA25758@lambchop.rdu.rpath.com> References: <453FD508.6010300@rpath.com> <4540AB01.8000804@rpath.com> <4540BE8F.3090309@rpath.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4540BE8F.3090309@rpath.com> User-Agent: Mutt/1.4.2.1i Cc: rmake-discuss@lists.rpath.com Subject: Re: [Rmake-discuss] Projects - a new way of using rMake X-BeenThere: rmake-discuss@lists.rpath.com X-Mailman-Version: 2.1.6 Precedence: list List-Id: Discussion regarding the rMake build tool List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Oct 2006 14:21:47 -0000 On Thu, Oct 26, 2006 at 09:56:31AM -0400, Jonathan Smith wrote: > Joseph Tate wrote: > > David Christian wrote: > >> Longer term, we're looking at adding the concept of "conditional > >> rebuilds", so that you can build a package (say, something that you > >> derive from upstream) if and only if something else you build requires > >> it. Basically, if you use rPath's packages, and you update one of their > >> buildreqs, we'd like rMake to be able to automatically recognize that > >> you are going to have to rebuild those packages. But that's longer term. > > > > Is that even necessary? It may be good practice, but unless something > > else has changed (different glibc or compiler), do you really need a > > rebuild if libjpeg has bumped without changing ABI? That's a lot of > > packages to rebuild. > > > > I think that you leave it to the developer to say whether this > > particular change merits rebuilding everything that depends on it. > > > > Yes, you do need to rebuild. If, say, libjpeg:devel is in the build > reqs, it is impossible for rMake to know what part of libjpeg is used - > could be the dynamic libs (which are actually part of libjpeg:lib), > could be the static libs (which *would* necessitate a rebuild) or could > even be some program who's behavior has changed. But Joseph is completely right -- the user needs to be able to control this behavior. That, as far as I know, is the plan; you explicitly add things to be conditionally rebuilt. If you don't add something to that list (directly or as a part of a group) it won't be rebuilt. If a user has inspected libjpeg and knows that there are no ABI changes, just a security fix, it's reasonable not to rebuild everything that depends on it. However, you certainly want to make it easy to deal with the case that the ABI has changed, which is one of the real life issues that conditional rebuilds are intended to resolve.