Survey of DAPNET use and implications as to potential member interest

[Andy Carson]

I was reading through from survey results from the Athol event and thinking about how useful it is to have a real measure public interest and what we can do to engage more people. It occured to me this morning that we have at our fingertips an excellent tool to determine the interest of the folks that frequent the website. I feel that if one is interested in a topic, one will eventually post about it. I understand this is a weakness of this survey technique I’ve used, but I think there is a lot to learn here none-the-less. I think we DO have the tools to understand what the public wants, and we SHOULD give them what they want, esp when we want to attract people. I will go through the categories one-by-one and give some comments as I go. I understand these data are going to be counter to what some might want to think the communities interest is, but I believe we MUST measure and listen rather than simply speculate. You will have to excuse the bluntness of my comments, but this survey is extremely clear and without much mercy.

Draft Animal Power Network comunication/Outreach
91 posts or 0.3% of total posts, indicating practically no interest
This area is dominated by conversations about DAPNet membership, we do have poor conversion of DAP.com’s to DAPNet’s

Our Member’s Map
69 posts or 0.2% of total posts, indicating practically no interest
This is neat, but not something people look at a lot

Draft Animal Power Front Office
373 posts or 1% of total posts, indicating practically no interest
Who reads the rules when they go to a website?

The Front Porch
4987 posts or 14% of posts, indicating moderate interest
Introductions, off-topic conversations, and diaries are popular here
The diaries have a strong content and education element
Many off-topic conversations are draft animal related, and have a strong content element as well
Introductions are as chummy as we get as an organization, and with good reason

Community of Interest
2834 posts or 8% of posts, indicating low-moderate interest
The events section dominates this category with listings of events to come
There is practically no interest in educational opportunities (This has been interpreted mostly as internships, with a sprinkling of off-site education -NOT on site education)
There is also practically no interest in books, lending, and terminology

Draft Animal Power
12486 posts or 36% of posts, indicating high interest
Particularly popular are the horse, oxen, training, and working sections
These are all areas of rich content of an educational nature

Equipment Category
5816 posts or 17% of posts, indicating moderate interest
The equipment section dominates this area, with detailed desciptions of tools and how to use them
Note that the equipment fabrication area is more than 5 times more popular than the buyers guide

Sustainable Living and Land use
5287 posts or 15% of posts
Sustainable Farming and forestry dominate this area, again with largely educational content
Note that there is practically no interest in draft animal history, use in the future, and skills and craft (although I think these areas are covered in different areas)

Marketplace
997 posts, or 3% of posts indicating low interest
It seems a low volume of animals/goods are sold here

Draft Animal Powered Forestry International
505 posts, or 1.4% of posts indicating very low interest
I think alot of these topics would also be covered in sustainable forestry, in the specific animal sections, or in the equipment section (I don’t think this apparent lack of interest means people aren’t interested in these topics in general)

Associated Organizations, Sponsors, & Collaborators
387 posts, or 1.1% of posts indicating very low interest
Particularly unpopular (with only 41 posts) is the section entitled “Clubs, Associations, Groups; Contacts & Links.” Even I, who is not really into this kind of stuff, am amazed at how ridiculously little interest there in this topic.

DAPNet BOD
992+28 posts, or 2.9% of posts indicating low interest
This is a meaningless number since it is only of interest to board members, but I included it for completeness.

I hope this survey provides food for thought. It is certainly eye-opening for me!

[Tim Harrigan]

Andy, are these threads or posts?

Another interesting statistic is how many members actually help support the web site and organization by becoming supporting members. I do not have that number off the top of my head, someone probably does. And at any one time the large majority of visitors to DAPNet are not members, but visitors. At least they are not logged in as members.

[near horse]

First – thanks for doing some sort of quantification. It is interesting and probably a good starting point for evaluating forum use. Just a thought or two – “number of posts” doesn’t indicate the “number of views” of a thread etc. I was surprised how many people rarely, if ever, post but still come here to read through what’s being posted by others. This isn’t unique to DAP.com as it happens on many forums and goes unseen until someone pops up and posts that they’ve been lurking for a long time etc. In addition, there is an “archival” element that allows folks to go back and reread posts on various topics (EX – ground manners). The value of that component might be underestimated if we just look at the number of posts.

[Andy Carson]

Tim, they are posts inclusive of all threads in the category. As some threads tend to wander, I think total posts is the way to go.

I agree, Geoff, and I thought some about if I should quantify number of posts or number of views. You are right that the two don’t always go together. The member map is the best example. Lots of views, few posts. Still, I was interested obtaining this data for the purposes of seeing where we could recruit members from. I think most members would first view, then post DAP.com, then join DAPnet (if they get this far). In my mind, we get lots of viewers so tracking things people “just look at” might be missing the point. I think it is better to look at the topics that not only made people look, but made them engaged enough to actually do something about (IE post). It would be better to have both pieces of data though, jsut to see. Perhaps I will add this when I get some time.

[Andy Carson]

Here’s a reworking of the numbers counting the number of “high profile threads” in each category. I arbitrarily defined “high profile” as threads that get more than 1000 views. Totally arbitrary, but there does seem to be a break at about this number between thread generally seen as “interesting,” and those seen as “uninteresting.” I wrote a bit after each set of numbers comparing this analysis to simply counting the number of posts. Most of the “take homes” are the same, but there is an important difference between the two techniques in the “marketplace” section

Draft Animal Power Network communication/Outreach
7 threads with >1000 views, or 0.3% of “high profile threads”
Most popular thread = 1816 views
-Identical result to “post count” technique

Our Member’s Map
1 thread with >1000 views, or 0.05% of “high profile threads”
Most popular = 7098 views
-Similar to “post count” technique

Draft Animal Power Front Office
51 threads with >1000 views, or 2.5% of “high profile threads”
Most popular = 10965 views
-This is more than twice as much interest as in “post count” technique. Apparently some people do read the rules! 🙂

The Front Porch
261 threads with >1000 views, or 13% of “high profile threads”
Most popular = 13061 views
-Almost identical to “post count” technique.

Community of Interest
185 threads with >1000 views, or 9.2% of “high profile threads”
Most popular = 18452 views
-Very similar to “post count” technique, a more detailed analysis shows a low level, but broad, interest in books that was not revealed in the “post count” technique.

Draft Animal Power
279 threads with >1000 views in horse section alone
637 threads with >1000 views, or 32% of “high profile threads”
Most popular = 83069 views
-Still the elephant in the room. 83,000 views… Wow! There are many “jackpot” threads in this category.

Equipment Category
264 threads with >1000 views in equipment category alone
359 threads with >1000 views, or 18% of “high profile threads”
Most popular = 13730 views
-Similar to “post count” technique. This category has fewer “jackpot” threads, but more of a general broad interest

Sustainable Living and Land use
274 threads with >1000 views, or 14% of “high profile threads”
Most popular = 15075 views
-Similar to “post count” technique.

Marketplace
201 threads with >1000 views, or 10% of “high profile threads”
Most popular = 13213 views
-This is over 3 times more popular than would be predicted by posts. Honestly, I would have expected the opposite pattern, where views reduce after the post gets old and most people think that the item is sold. Perhaps this means that the marketplace is just not a place that converts “readers” into “posters” as much as other categories do? Perhaps people simply want to do thier buisness privately? Perhaps there just isn’t much to say about someone selling something? Either way, it is nice to see that the marketplace does attract a large number of views, even though it might not generate lots of posts.

Draft Animal Powered Forestry International
29 threads with >1000 views, or 1.4% of “high profile threads”
Most popular = 10460 views
-Identical to “post count” technique

Associated Organizations, Sponsors, & Collaborators
11 threads with >1000 views, with 0.5% of “high profile threads”
Most popular = 3736 views
-Similar to “post count” technique

DAPNet BOD
0 threads with >1000 views (no “high profile threads”)
Most popular = 235 views
-Similar to “post count” technique

[Andy Carson]

Geoff, when you mentioned counting views rather than posts, I was worried that this would select areas that were somewhat “sensational” and might get the attention of “passers by” rather than really engaging interested invividuals. The most popular thread, with 83,000 views, is an old link to a video of Neil Dimmock driving 26 horses, and certainly fits into this category. When you get past this post, though, it really gives you a lot of respect for the people that this site attracts and the degree to which they are attracted. The next most popular post is about using a D-ring harness on a donkey. This is an extremely specific topic that I doubt one could find anywhere but here. The next most popular is a electrolyte recipe for scouring calves. This is not “light and fluffy” reading material… Next is a discussion of horse progress days. It is interesting to note that there are several posts about horse progress days on this site, but this one was elevated to “superstar” status, while the others recieve only a moderate number of views. There are only 6 posts on the thread, too. The posts themselves say both good and bad things about the event (something this site is so great at!), but other threads about HPD seem just as in depth. This one is a mystery, but I think we should look for patterns in these data rather than try to explain specific threads… After the HPD thread, the next most popular is the discussion of draft buffers. I am somewhat susprized by the popularity of this thread, as I felt that even as I was contributing to the thread I worried it is was going to be nerdy, boring, and too detailed for a large audience. Clearly this was not the case. Here’s the complete list of the top 25 most popular threads, with number of views per thread (x1000). I will leave it to the reader to see patterns…

Check me out (Neil Dimmock video driving 26 horses) =83.1K views
D harness on a Donkey =31.6K views
Electrolyte recipe for scouring calves =22.7K views
HPD (one of several threads discussing Horse progress days) =18.5K views
Draft buffers =18.3K views
Training Them Old School =16.3K views
The future of the dairy cow?? =15.1K views
Odd Jobs =14.9K views
No.7 McCormic Deering Mower =13.7K views
FSA loans =13.2K views
Harnessing the Powers of Youtube for Good =13.1K views
Hi from Essex Farm =12.5K views
The Bakery Wagon =12.2K views
Haying Techniques with Draft Animals =12.0K views
Plans for forecart? =11.6K views
NAIS is not dead…. =11.5K views
Poll: Tragedy!!! Includes discussion of dramatic experiences, and blinders vs. open bridles =11.4K views
Cost of Maintaining DAP =11.0 K views
New Saw? =11.0K views
Driving oxen with lines indian style:good or bad? =10.9K views
What Do You Feed Your Horses? =10.6K views
How many horses =10.6K views
Hybridized Timber Harvest – Horses and Fowarder =10.4K views
Elephants vs Horses? =10.2K views
Fuel Prices…. =10.0K views

[Carl Russell]

Talk about rabbit holes…… Pretty interesting stuff, but it is important to realize that most of the visitors (views) to this site are robots…. google spiders that search down certain terms, names, content, or recent activity, etc…… Which ultimately is important for us, as that helps to elevate DAP.com on the google search which leads people here, but may not translate to any usable statistics for evaluating content.

Most of the categories, and threads have developed from interests expressed by visitors here. So, while there is some value in trying to figure out what most people are looking for most of the time, one value of this site has been that even obscure interests could/can be explored.

The “atmosphere” of this site, the open, inclusive, respectful tone is as valuable to visitors here as the actual content. The content is important, but if folks don’t feel comfortable asking, or posting, then content would not have grown like it has.

I found when I was administrator that interest grows where interest is placed. If you were to seed the clouds in an unused category, I am willing to bet that you will see activity there. I used to look for specific topic areas within posts, create a category for threads like that, and then activity would follow.

Andy I am excited that you are taking such an interest in this site. Your analysis will only benefit our shared interests.

Carl

[Andy Carson]

I am glad that you can see value in this survey, Carl, and I think you are right to be sceptical about looking at “views” rather than “posts.” Overall, the data trends are the same between the two methods, though, which validates both methods. I like to analyze data in lots of different ways and when the data are consistant, you can trust the results more.

I am certainly not suggesting that unpopular categories be eliminated, nor do I think this is wise. I am partly using this data to determine areas of interest, as well as the dpeth of knowledge people are interested in, in plan possible videos, events, or other enhanced in depth features that we might include in the future. For more expensive and time consuming things, I think it is wise to focus on the areas of highest impact. I think this analysis, along with in person surveys, will help provide guidance and is a better way to go forward than pure speculation. I do not think it is a substitute for speculation and thinking, but I believe it should not be ignored either.

[Andy Carson]

I do appreciated everyones comments on this analysis. Addressing these concerns has given produced 3 different ways of looking at the numbers, and I will now present a fourth and fifth way that address some of these concerns. In this analysis, I have tracked the number of “high profile threads” in each category. In the first analysis, I have tracked high profile threads by category, with “high profile” threads defined as those that have generated 20 or more replies. Again, this is an arbitrary break, but it does generally correspond to thread generally seen as “interesting,” and those seen as “uninteresting.” As posts are definately generated by people, not robots, this eliminates this particular concern. After this, I have listed the “jackpot threads” as defined by number of posts. I put a star (*) next to any of these posts that are also listed as “jackpots” by counting views. There is a lot of overlap (64%) between the two methods, and alot of the “most random” threads are not found here. I think by using two methods to screen, you remove alot of the “randomness” in the system. I believe this goes a long way towards addressing the “rabbithole” aspect of using views alone to define “jackpots.” I will leave the reader to find patterns for now.

Draft Animal Power Network communication/Outreach
0 threads with 20+ replies, 0% of “high profile threads”
Most popular thread = 19 replies

Our Member’s Map
1 threads with 20+ replies, 0.3% of “high profile threads”
Most popular thread = 62 replies

Draft Animal Power Front Office
2 threads with 20+ replies, 0.6% of “high profile threads”
Most popular thread = 45 replies

The Front Porch
34 threads with 20+ replies, 11% of “high profile threads”
Most popular thread = 75 replies

Community of Interest
12 threads with 20+ replies, 3.8% of “high profile threads”
Most popular thread = 48 replies

Draft Animal Power
134 threads with 20+ replies, 42% of “high profile threads”
Most popular thread = 196 replies

Equipment Category
52 threads with 20+ replies, 16% of “high profile threads”
Most popular thread = 92 replies

Sustainable Living and Land use
69 threads with 20+ replies, 22% of “high profile threads”
Most popular thread = 117 replies

Marketplace
2 threads with 20+ replies, 0.6% of “high profile threads”
Most popular thread = 43 replies

Draft Animal Powered Forestry International
6 threads with 20+ replies, 1.9% of “high profile threads”
Most popular thread = 117 replies

Associated Organizations, Sponsors, & Collaborators
6 threads with 20+ replies, 1.9% of “high profile threads”
Most popular thread = 25 replies

List of “Jackpot” Threads with 80+ replies. Remember, I have starred (*) posts that are also “jackpots” by views. 64% are cross listed, and are clearly interesting by all measures.

Draft buffers* = 196 replies
Chicken predator ID = 130 replies
The future of the dairy cow??* = 117 replies
Odd Jobs* = 117 replies
Hybridized Timber Harvest – Horses and Fowarder* = 117 replies
Training Them Old School* = 116 replies
D-ring Harness Origins = 92 replies
The Bakery Wagon* = 91 replies
Fuel Prices….* = 89 replies
Scoot hardware = 85 replies
White bird play day = 84 replies
Haying Techniques with Draft Animals* = 83 replies
Logging workshop hosted by DAPNet and The Farm School Athol MA = 81 replies
Poll: Tragedy!!! Includes discussion of dramatic experiences, and blinders vs. open bridles* = 81 replies

[Carl Russell]

@Countymouse 37866 wrote:

……..
I am certainly not suggesting that unpopular categories be eliminated, nor do I think this is wise. I am partly using this data to determine areas of interest, as well as the depth of knowledge people are interested in, in plan possible videos, events, or other enhanced in depth features that we might include in the future…….

I guess I wasn’t so concerned about how to manage the site, and was aware of your overall objective. I was just suggesting that there are other factors that go into developing an active thread….. take D-ring Origins, or Draft Buffers for example….. which in my mind are pretty obscure topics…….. but the thoughtful discussion, and inclusive nature of evaluating differing views or criticisms, make for very attractive reading and a welcoming participatory environment. I also think that specifically in those cases there may have been a lot of posts, but many of them are by a small number of people offering a lot of content.

All in all I like your approach. I think there are a lot of topics in this field of interest that people don’t realize that they are interested in, or don’t know that they should be interested in, or maybe they don’t even realize that an issue exists. Taking the time to make informational presentations on important topics will be more valuable in the long run, than just making presentations that respond specifically to expressed interests.

You are one who has a critical mind. I think that you come up with very interesting ideas that you end up opening other peoples eyes with…….. just bare that in mind.

Carl

[Andy Carson]

@Carl Russell 37876 wrote:

I think there are a lot of topics in this field of interest that people don’t realize that they are interested in, or don’t know that they should be interested in, or maybe they don’t even realize that an issue exists. Taking the time to make informational presentations on important topics will be more valuable in the long run, than just making presentations that respond specifically to expressed interests.

I agree completely. I think it would be uninteresting and unwise to simply regurgitate this same subject matter. I think the best content would be to present new information that “fits in” with these popular posts. Determining if something new “fits in” is always an intellectual challenge, but I think this survey gives us a mental framework that I think is helpful.

[dominiquer60]

Well done Andy, It is nice to see all of this information presented with different factors considered. It is great that you have taken interest in this site, it is DAPNet’s gateway to the rest of the world and has the potential for education that the newsletter and live events can’t offer (though we should continue them). I am looking forward to what we do with this compiled info.

[Andy Carson]

@Carl Russell 37876 wrote:

the thoughtful discussion, and inclusive nature of evaluating differing views or criticisms, make for very attractive reading and a welcoming participatory environment.

A couple days ago, I start monitoring activity around a specific thread as an experiment and I want to share the results before they get convulted by the present of video content.

Tom posted an update to an old thread about draft logging research. It had about 2000 views and 22 posts at the time. It had been up for almost a year and a half (first post was 7-10-11), but that last post (before Tom’s) was over a year ago (8-02-11). This gives a year to see how many “views” the subject matter itself gets. It was of moderate interest, but not a “superstar” thread. So I thought I would get involved partly because I was interested and had not shared thoughts on this subject yet, and partly because I wanted to test Carl’s theory (above) about what really makes “superstar” threads so attractive. Quickly, other thoughts were contributed by a host of people and the thread snowballed into something interesting to everyone. It’s over 6600 views and 52 posts now. In two days, the thread attracted more than twice the views as it was able to in a year and a half. I know I am not the only reason that this thread got popular, and there was active discussion before I joined in. My feeling is that my participation, as well as that of others, brought this thread past a “tipping point” where readers recognized that there was broad diversity in the thoughts presented, as well as an open and inclusive exchange of these ideas, and could see the passion that some people have. I think this is proof that it is open exchange coupled with diverse backgrounds and opinions, that elevates threads to “jackpot” status. It is not the subject matter itself, but rather the format and the diversity of viewpoints.

Oh, by the way, in case anyone thinks that I was “making up” opinions in order to get a conversation started, rest assured I was not. I will admit I occasionally do this, but it rarely actually starts useful discussions, and there are plenty of honest differences I have with others that it is unneccesssary to make additional ones up. We probably all have different ways of thinking, and this experiment is good proof that contributing your personal thoughts to the open discussions facilitated by this website is interesting to wide audience.

So, you were right, Carl. It is not the information itself that is attractive, but our diversity of viewpoints and our open, thoughtful, and passionate intellectual exchange. I just had to experiment a little myself to verify this. It’s my way…

[Tim Harrigan]

@Countymouse 38193 wrote:

… It is not the information itself that is attractive, but our diversity of viewpoints and our open, thoughtful, and passionate intellectual exchange. I just had to experiment a little myself to verify this. It’s my way…

That is true, Andy, and there are a lot of ways that this information could be viewed. Membership and awareness of this site has grown considerably over the years so there is a bigger audience now than when that thread first started and ended. Also, when you revive a thread it becomes active and becomes highlighted on home page until it has been dormant for 24 hours. Many of these views may be quick clicks with little engagement based on lack of interest. So maybe a more important number is the number of view that occur when a thread is not active. Those folks are actually looking for something. To them, the most valuable gems might be some of the most obscure with few views.

Others numbers that I quickly approximated: Only 7% of the listed members have more than 10 posts. Only 8% of members are currently classified as active. 2.3% of the currently active members account for about 80% of all the posts. So if the interaction is important, why don’t more folks interact?

[Andy Carson]

It is good to have more numbers. According to compete site analytics, draftanimalpower.com gets an average of 4,011 unique visitors per month. Counting unique visits is one of the most established ways to determine impact. This 4011 visitors is far in excess of the 358 active members. My conclusion is simple, 90+% of visitors do not register or log on when they visit.

How does 4,011 unique visits stack up? MOFGA.org gets 13,921 unique visitors per month, so we are about 29% of this impact. Perhaps this seems poor, but given the broader focus of a MOFGA.org, ~30% is great. Compare this stat to any other stats between MOFGA and DAPNet, and you will see DAPNet is a vastly smaller organization. MOFGA.org has 5000+ members, 2000+ volunteers, 31 paid staff of various types, hosts 20 small educational events per year, and thier large event attracts more than 50,000 people annually. Taken together, the number of visits we get is much LARGER than the number of visits one would predict based on the size of our organization. I think this is something especially good that our organization does.

[Author]

Posts