Scrivener and the Fleece & Fiber Sourcebook

—> Long post warning

Before I launch into a fairly technical discussion, I want to mention one of the things that makes it all worthwhile. This week's Spinning Daily newsletter shows what Betsy Alspach did following the SOAR workshop that she took with me this past October. Thanks to Betsy for continuing her enjoyment of the fibers we played with; for making her magic skein; and for sharing it and the story. Click the image to read the whole article online.

_____

The nitty-gritty

I've been researching and exploring new software solutions. As I do so, I would like to shine a spotlight on a piece of software that I came across by accident, as a result of a series of computer disasters. I treasure it. It is, and will continue to be, key to my work.

When I began research for The Fleece & Fiber Sourcebook, I employed PCs running Windows XP and, the primary machine for that project, a laptop running Linux. I organized my research in traditional computer folders. The method was cumbersome, but familiar. I did a lot of tedious keyboard work—for example, making notes from websites—while remembering, and being glad to be past, the days of writing longhand on index cards.

About that time, the PCs started not playing nicely and I lost about two years' worth of income-producing work: not because my hard drive crashed or my files were lost, but because all my time (and a lot of money, hiring consultants) was spent trying to get the PCs to let me get something accomplished other than talking to tech support.

Cutting to the chase, after waiting far longer than I should have (because although the solution cost a lot, so did the troubleshooting), I ditched all but one of the PCs, bought a couple of Macs, and . . . discovered

I like Scrivener so well that even though time now is short, I opened up Illustrator and played with some type in a garish display of enthusiasm. (That's P22 Vale Pro, a dignified face that I've abused amused(?) with an array of colors.)

In 2011, a version of Scrivener became available for PCs; in 2008, it was Mac-only. If I'd known about it, I would have figured out how to buy a Mac just to run it. Wow, did it make my life easier for the extended, focused, broad-based, complex project that became The Fleece & Fiber Sourcebook. I'm not sure who told me about Scrivener: my niece, Corey, or my daughter's best friend, Jenny, or both. They, and the program, saved whatever shreds of sanity I had left after the PC nightmares.

This will be a nuts-and-bolts post about organization and process. Keep in mind that I was using an earlier version of Scrivener. I've upgraded, but I haven't implemented any new tricks in this legacy file. I also didn't master all of the software's capacities in the version I used then. I didn't have to. Without asking a lot of me, Scrivener provided a flexible, friendly, reliable foundation for a huge project.

_____

The Scrivener screenshots should be clickable for larger versions. I generally try to keep my images small, but because some people may want to see more detail and I still don't want to hog bandwidth for my friends who are on dial-up (yes, some are), I'm trying something new.

_____

Moving into Scrivener

I'll start with my overall organizational scheme. I'll drill down into it, then back up to show part of my working process. The entire process would require another book to explain. . . . I hope that what I'm presenting here is helpful and clear.

Keep in mind that you're seeing the final working file. It evolved over several years and through many iterations. Scrivener is so accommodating that way. I would save a snapshot (so I could go back, if I needed to), move things around, and keep making forward progress as I did so. I never did have to go back to a saved version. And I should mention that one of the things I like about Scrivener is that everything in it can be accessed without the program, should computer disasters occur. I don't find a link explaining that detail right now, but I remember it clearly. Scrivener's storage methods are not proprietary.

I started by importing my existing PC folders full of files into Scrivener, which was easy, even though there were heaps of them. Then I started arranging and rearranging, as well as continuing with my research and doing things I couldn't do before—like importing web pages instead of re-keying the pertinent bits.

The Draft folder: overall conceptual framework and a container for final files

In Scrivener, a Draft folder can contain only text, not images or other miscellaneous file types. So I used that top, Draft section for an overview of where we were going, and then as a container for final files. The heavy lifting took place in the Research folder.

This is the big picture, the view from space. Within each folder, we estimated the components' final page counts. The associated file for each family or group sketched out how we intended to use those pages. The allocations changed, of course, but you see the "introductory section" above was scheduled for about 28 pages.

I also ultimately made a second Scrivener project file for Fleece & Fiber. The original project collection ended up being dedicated to the sheep, with cross-references to the second file for all the other animals and the peripheral research (synthetics, silk, and plants). I did keep the old folders in the original file, mostly because I didn't need to take them out (speed, speed . . . ).

Here's the breakdown for the sheep section:

At the bottom of the sheep folder, there's a subfolder called "Sheep reference files for proofing," which contained the text-only versions that we compiled into the final manuscript. I worked FROM this area in Research down into the Draft folder, and then brought my finished work back up here.

The next view shows both the organizing pane on the left and a portion of one of the component files in the window on the right. I've highlighted the file for the English Longwool family. At the right, you can see what breeds we planned to include in this section and the estimated number of pages for each (the bottom showing some of the other breeds we needed to keep in mind as we prepared this section, even though they aren't English Longwools):

A good six months' worth of work preceded the establishment of these folders, during which co-author Carol Ekarius and I went back and forth through e-mail, Skype, and in-person meetings about what would be included and how we would organize it. That's where the "family" and "group" units originated. Sheep breeds can be sorted and associated with each other in infinite numbers of ways. Our final method balanced my fiber-related wish list with Carol's livestock-focused perspective.

I haven't checked yet to see how closely the finished book conforms to the listed page-count targets. Everything changed constantly as we proceeded. (Silk and synthetics obviously weren't part of the scope of our finished work. We pulled them out in the interest of completing the book before the turn of the 22nd century.)

The Research folder: where I lived

Unhampered by the sensible yet strict limitations of the Draft section, a Scrivener project's Research area can contain many types of files: text, images, imported web pages. . . . Because of its flexibility, the research section of the Fleece & Fiber projects (numbers 1 & 2) became my electronic home for several years.

Here is the highest-level view of the sheep-corral Scrivener project, with the research subfolders revealed. It's deceptively (and comfortingly) simple.

Yarns contained details about potential sources for sample yarns (as opposed to fibers). That was mostly Carol's department, although when I came across options while I was doing other research, I'd make notes here and then tell her about them.

I'm going to demo with the SHEEP research.

To make the second project for the OTHER MAMMALS, I copied the original all-encompassing file, renamed it, and deleted all the sheep info from that copy. The X and Z codes in the folder names are my signals that this is information peripheral to the current project, although it's stuff I want to keep track of.

Going deeper into research

Now we begin to get serious. Here's the next level of files and folders under SHEEP research:

At the top are templates that I used for specific types of files, and a couple of running bibliography files. Scrivener now has (and may have had then) better ways to manage templates and bibliographies. My manually implemented methods worked fine, although I pushed the limits of practicality. For the next big project (she says, with trepidation), I'll do more with templates and corkboards. I really didn't need them this time.

Here's a quick side trip to see the contents of my templates folder:

Back to the main tour.

I organized the sheep breeds (and types) alphabetically in subfolders. Initially the folders encompassed larger chunks of the alphabet, say A through F. As time went on, I had to slice more finely. The S breeds ultimately needed two folders!

Here's the C folder, which I'll use for the demo because there were enough breeds to make a one-letter group but not so many I can't get them into a screenshot of reasonable size.

My comprehensive list included breeds that were mentioned in sources I consulted but that have become extinct. (The x in front of the name reminded me not to pay too much attention once I had resolved the breed's status: having these records within the file was useful when I was tracing the development of other breeds that still exist.)

A few of the breeds in that list did not end up in the finished book because we ran out of time. If we had fiber, we fit the breed in. If not, we were forced to let some go. We discussed mules as a category (for example, Cheviot Mule), but mules are crossbreds so the many types of mules didn't get individual treatment.

There are also cross-references in this list. CVM ended up being filed under R for Romeldale, because CVM is currently being treated as a specific type of Romeldale and not as a separate breed (there was a time when it was classed differently). To keep myself from wondering whether I'd lost my CVM notes, I made a blank file to tell me where I'd put them.

Clun Forest is highlighted in the screenshot because I'm going to use it as my example in drilling down farther. Within the Clun Forest folder, which is shown in its finished form, are the DRAFT (working text), a SAMPLE (spinning notes), BREED NOTES, which contained my summary of my research, which is in the stack (rather than folder) of Resource files. The files within the folder progress, in level of completion, from bottom to top:

resources (research) —> notes (summarizing) —> sample(s) (spinning) —> draft (writing).

All of the files and folders contain the breed name, so if things got out of order I could put them back. Items moved constantly as I was working, and this naming convention served as a safety net. I added notes to the FOLDER names about the level of completion for that breed, so I didn't have to open the folders to see where I needed to turn my attention next.

EVERY BREED had a set of files that looked like these. Many sets were much more extensive.

The Resource files got the major action at the start.

I had a template file that stayed at the top of the resource pile called RESEARCH LIST. It summarized the basic resources that I wanted to be sure I checked for each breed. If there was no information on the breed in that resource, I'd mark "none" so I'd remember that I'd already done it. For example, there are no notes for the Clun Forest from Ryder (Sheep and Man). Because Ryder was a primary resource that I was unlikely to skip, yet the information in it is diffuse and requires time to assemble, I might think this was an oversight. I could check my research list and discover it wasn't. From here on, the screenshots show the folder organization on the left and part of the view of the highlighted file itself on the right. Scrivener, of course, has a larger screen area and shows more of the file. I've just cropped my images to keep this discussion as nimble as possible.

I had, indeed, checked Ryder. Yes, he had something to say about Clun Forests, but not enough to warrant a whole file of notes. (Often my Ryder notes were extensive.) On the summary page, I recorded page numbers and topics, in case I needed to retrieve a specific piece of information later.

"Dohner (and mixed sources)" above means that this was an imported file from early research and I'd keyed information from several locations into a single file. Once I was in Scrivener, I did separate out the Dohner portion, because the book (Encyclopedia of Historic and Endangered Livestock and Poultry Breeds) was one of my most important sources. (There's an individual file for it later in the list.) (Scrivener handles multiple files much better than my old C:/sheep folder did, and lets me look at the contents, not just guess at them based on the cryptic names I've given them. Oh, and its search functions are superb!)

The point was not to maintain the system perfectly but to set up scaffolding strong enough to support the bigger tasks of research and writing.

Once I had completed the review of resources, I compiled BREED NOTES into which I cut and pasted (or typed my comments about) the highlights from my searches. This file consisted of quotes and summaries, and is what I put in front of me when I actually started to write (using Scrivener's split-screen function, also a boon at many other times).

See the note about "Data transferred to Numbers" at the bottom of the righthand pane? While Scrivener doesn't handle spreadsheet information especially well, I managed to put some spreadsheets into my BREED NOTES files and to manipulate some data there. I supported this with Numbers (Mac spreadsheet) files; that's part of the "how did I come up with the numbers?" discussion. (Another idea on my list of topics to write about here is how I decided what numbers to put in the breed data boxes for The Fleece & Fiber Sourcebook.But that wasn't a Scrivener task.) 

When I spun fiber from the breed, I made succinct notes about the samples, the tools and techniques I used, and my subjective impressions. I kept a number of the variables standardized, for speed and ease of record-keeping, so those items aren't in the notes. Unless otherwise specified, I used a Lendrum folding wheel with a standard flyer, on its highest-speed whorl.

For some breeds, I made a lot of samples. I chose Clun Forest for this demo because it has all the necessary parts but was not complicated. Each SAMPLE file refers to one source or color of fiber—i.e., one fleece or batch. If I made more than one skein from the same wool, all my experiments are in the same file. (My sample notes are backed up by a separate database (print and electronic) of the fibers we obtained, where they came from, and what they were like. There's enough information here—Hagerstown, MD—for me to locate more detailed background on this wool.)

Finally, I would write. That step occurred in a DRAFT file (still in the Research section, not the Draft section).

When I finished, I would send a copy of that file to Carol, because this was the point at which our writing process became collaborative.

When I sent Carol a file, I marked its status in another Scrivener record I made up to track where we were in our final coordination on each breed, because we sure didn't work in lockstep—as individuals or as a team—from page 1 to the final page! Our major collaborative activities occurred at the beginning of Fleece & Fiber, with a determination of what we would cover and how it would be organized, and at the end, when we blended our contributions into what we think, and hope, is a seamless whole. In between, we kicked ideas and tasks back and forth as needed, but worked independently for the most part.

I put this tracking file under "Templates," back up at the top of the Research folder so I could find it easily.

I still kept within this list the names of breeds and types that we were not covering. Having that information readily available saved time and sanity. In an undertaking of this size, the less time you spend hunting for or wondering about things, the better.

At the bottom of the Templates group, you'll see a TIME PERMITTING summary. This was a place for me to note tasks I'd thought of to do, if we had time: additional types of samples to spin from specific fibers I'd had to set aside in order to move to the next; sidebars we might have written if we had possessed infinite time and could have printed infinite numbers of pages; and so on. I did complete the most important tasks on the list.

And there is, of course, the whole second Scrivener project file for the Other Mammals, which ended up including the EXTENDED sheep as well.

_____

Scrivener, my stalwart companion, has requested some software allies

Scrivener is an amazing tool for organizing and writing a massive project. Carol and I also used a lot of spreadsheets. The array sufficed. We made it.

So why am I looking at other software solutions?

I'm asking Scrivener to do jobs it wasn't designed for. It's been doing them exceptionally well to date, but I need to reserve it for what it's best at: organizing and facilitating writing projects, large and small.

Scrivener is fantastic when you know what you want to do. There are other tools that will work better for the amorphous, preparatory stages, and for managing information that may be used in multiple projects, not yet defined. It's also been fine for organizing workshops so far—planning, tracking details, and putting together the materials and handouts—but as I teach more workshops, I need to quit asking Scrivener to bend over backward for me.

I need to supplement Scrivener with a DATABASE to track fibers, yarns, and workshop information. I need a dedicated BIBLIOGRAPHIC system. I need a way to save, sort, organize, and OCR the PDFs and notes I'm gathering, even though I'm not sure which project or projects I'll need them for. And I need a flexible technique for gathering and moving my ideas around before they've become projects, so I can see how my coming-into-focus thoughts relate to each other (by means of a tool that won't let my ponderings get crumpled up and lost in a pile of papers).

I'm currently beginning to learn:

• Database: Bento (the learning curve and price are appealing, and it already seems helpful)
• Bibliographic system: Bookends (coming along nicely, although I'm not yet sure how it will connect to the other software—actually, the matter of connection is a question I have about everything I'm doing)
• Collecting and organizing research: DevonThink (I have it, and have read the Take Control book and a bunch of stuff on the forums)
• Ideas: TinderBox (I have it, and have read a bunch on the forums and have loaded the tutorials onto my computer)

This is going to take a while. Researching, trying out, deciding on, and investing in the tools has taken about a year. Now I need to get conversant enough with them to entrust them with parts of my workflow. I also have deadlines, which would be easier to meet if I had the new systems in place. Yet the time I can dedicate to learning them is limited.

Onward. The eternal balancing act.

_____

I know this: Scrivener is solid. Even though I've discovered I can't do everything with it, I can do amazing amounts. It's gotten me this far. And Scrivener is a tool that makes work fun.