Update November 6th, 2015: wpDiscuz 3 has been released, check our review below.
WordPress’s built-in commenting system is pretty weak. Comments have been left partly crippled as Automattic used to have a horse in the third party commenting race with its own Intense Debate technology. Our creative director doesn’t like third party scripts for a few reasons:
- pageloads are very slow
- you lose the value of user generated content in most cases
- you are giving away quite a bit of your visitors’ privacy (important for political sites)
Indeed he doesn’t like real time commenting at all as it slows down page loads and increases server load radically. Regardless, some of our clients who use Disqus now would like to move to native comments but retain the real time and voting features of Disqus. As we are here to make clients happy, I will go through three of the more popular commenting options in order, sticking to technical issues related to loading speed.
- Disqus: the elephant in the room and why native solutions are coming strong now.
- wpDisquz: the mature drop in Disqus replacement but with some of the the same loading issues as Disqus itself.
- Epoch: the brand new open source sister of Postmatic comment notification and reply by email service. Supposedly lightweight and fast loading.
Disqus
Disqus is very popular right now as it offers unified login for commenters across websites and chat-like experience. However it’s an external service which puts a lot of JavaScript on your website (and causes a lot of HTTP requests – I’m seeing 33 for empty comment thread), so we don’t recommend it.
Here are the alternatives which operate solely on your website:
wpDiscuz – WordPress Comments
We were happy to find this plugin as it allowed some of our clients to stop using Disqus to get realtime comment updates. It looks like a complete package and it really does a lot of things right. For example the comment posting dialog looks very simple and you get more fields once you click into it.
wpDiscuz integrates well wit any theme
We don’t like Captcha, but there is an option for it as well as a simple email comment subscription feature:
wpDiscuz – posting a comment
It also integrates with social login plugins:
- WordPress Social Login
- Super Socializer
- Social Connect
- Social Login
Unfortunately the plugin doesn’t allow moderation of comments in front-end so we wanted to put support for it into our front-end comment moderation plugin FV Thoughtful Comments.
While registered users can edit their comments and vote on other users comments the message for a pending comment looks very basic:
wpDicsuz – pending comments don’t show up in the list
We also don’t like the comment paging which goes by parent comments only (we set it to 3 comments per page as the parent comments often have a lot of replies, but in some cases they don’t, so it was hard to get through all the comments). Also direct links to individual comments are not possible as it won’t load the required page.
We also don’t like comment paging at all, but the plugin takes 22 SQL queries to show 6 comments and the JavaScript code is slow as well – we had to tweak it to allow real use of the plugin. Plugin authors promised to fix this: link to wpDiscuz WordPress.org suppor forums
Live comment updates work fine, we just blocked them for users who haven’t left any comment to save our server resources.
Aesthetic issues/touch and feel: our creative director considers wpDisquz very ugly in its native guise, requiring a lot of custom CSS just to dress up adequately to be allowed admission to a website.
You can get it at wpDiscuz on WordPress.org
Update: wpDiscuz 3 just got released
Good news is that wpDiscuz developers listen to their users and they have released a new version which improved the following:
- Comment paging – the paging now works like in classic WordPress and you can link to individual comments!
- Custom sorting – you can sort by latest, oldest, most voted, just what we need
- Guest users can be excluded from live updates – good for lowering the server load on membership websites where only logged in users participate in discussions
What they didn’t improved:
- Number of SQL queries – we see no change although they promise 10x less queries.
- Speed – they promise 10x improvement, but we only an improvement of around 50%
- Styling – all the icons used in interface are inserted as images – making the size of HTML much bigger.
As you can see their claims about speed are not entirely true. But to be fair I did one more comparison of the performance of bare-bones WordPress comments with wpDiscuz 2 and wpDiscuz 3 on a cleaner website with 1,000 comments, as that’s were wpDiscuz 3 should show improvements – large number of comments.
The numbers are a bit surprising:
| Seconds | SQL count | |
|---|---|---|
| WordPress, no plugins | ~3.8 | 34 |
| wpDiscuz 2 | ~2.9 | 42 |
| wpDiscuz 3 | ~7 | 2056 |
…and here is the explanation:
- WordPress is already slow even without plugins, we fix that by FV Thoughtful Comments which really IS much faster as it stores old comments as HTML
- wpDiscuz 2 is actually faster than WordPress alone. I haven’t realized that as on our website all comments are posted by WP Users and that seems to slow it down a lot (in comparison to FV Thoughtful Comments)
- wpDiscuz 3 looks like the slowest one here as it does all the SQL to figure out comment parents/children – the plugin tries to cache this into wp_commentmeta but it must be failing for us. I guess if this works for you, it can be faster.
So my recommendation to wpDiscuz team would be to:
- improve working with a lot of users (just pre-cache the user data matching the user IDs used in the comments that are being loaded)
- and also examine the full comment list in PHP to determine which comments have replies rather than using SQL queries and some extra caching for results.
Then the plugin should indeed be faster.
Epoch – A native Disqus alternative with a focus on speed and privacy
The plugin has nearly no settings, we like that. Seems like it has issues integrating with some of the templates as there are multiple options of how it should be inserted, which is confusing – the “aggressive” option uses iframe.
I figured out there is a conflict with FV Antispam, once I disabled it the comment posting and comments autoupdating started working, so we should fix FV Antispam to work with it.
Epoch – list of comments – users see a nice clock icon next to pending comments
Direct links to individual comments work, so that’s great. However there is no paging, so if you have 1,000 comments on some post, it takes 5 seconds for the comments to load from server (read below about Ajax) and then another 15 seconds of JavaScript!
Admins get “Trash”, “Spam” and “Approve” links for quick comment moderation right there on the post page – similar to what we provide in FV Thoughtful Comments.
Epoch – comment moderation in front-end
The live updating works. However if there is a new pending comment, you won’t see it coming in when you are logged in as admin. You have to reload the page to see it. And as the user who posted the comment you also have to reload to see if it’s approved or not. I hope it’s just a bug and the developers can fix it.
While it says it’s focused on speed we have to report that it’s a real waste of resources. The plugin takes 3 WordPress Ajax requests (any new Ajax request makes the server nearly as busy as another pageload as it has to load the database and run PHP etc.):
- first it checks if the comments are open – 0.4 seconds
- then it checks the count of the comments – 0.4 seconds – this also keeps running each 15 seconds (default value) to do a live check on new comment
- then it loads the comment HTML – 0.5 seconds, or even 5 seconds if there is a lot of comments
- the comments are not in the page HTML!
- all of this happens even for guest users, even if they are not allowed to comment
Have look at all the Ajax requests – first three are required for the comments to show up at all, then there is the periodical live update checking for new comments
wpDiscuz has a big advantage here, it includes the comments in page HTML and the Ajax occurs only for live comment updates (0.5 seconds or more if you have a lot of comments and don’t set the paging – but at least you have the option unlike Epoch):
wpDiscuz doesn’t overuse the Ajax
That said, when posting a comment using wpDiscuz it takes more than a single Ajax request, while in my opinion it could just use standard POST, so there is room for improvement of wpDiscuz as well.
Email subscription is just not there and you have to use Postmatic - a service which allows comment email subscription and also reply via email. There is also a paid option – we don’t mind paying for it for some bigger websites if it works well, so we will see if the performance and moderation bugs are fixed. In fact the beta pricing at $9/website/month is a steal for any website. The email reply works really well and the emails look great.
Much better looking out of the box than our normal $10 one time license choice Comments Plus from Satollo (which also includes a simple image upload which is a real boon, exactly the kind of innovation missing from native comments). Email notifications can be customised so for a larger site building a better template is not a barrier but for a simple self-run site a good looking out of the box template really helps. Not having to deal with email server issues (we use SendGrid SMTP service for service email as managing bounces and spam reports in-house is too much for even a medium sized agency these days).
Aesthetically our creative director rates Epoch a win as it looks like native comments so it’s easy to style and many styles will just carry over.
Neither Epoch nor wpDiscuz provide user configurable option for sorting by date or comment rating and we are asked for this a lot.
You can get it at Epoch on WordPress.org
Conclusion
There isn’t a clean winner right now. Disqus warts and all does have the momentum of a huge installed user base of registered commenters. As a third party service, it’s unlikely to ever get really faster though. Disqus can hardly take away features at this point (although perhaps it could make some of them optional).
wpDisquz has squandered early momentum by a non-scalable architecture, slow development and poor aesthetics. At the technical level, Epoch is fairly poorly written now with even worse scaling than wpDisquz.
There’s space for either of these contenders to take over the real time native commenting with concentrated development. There’s still room for a third contender to come in fast and hard focused on performance and even charging for the basic or a pro edition (freemium model). Publishers who need/want this kind of advanced commenting system have money to spend. We have our video player to improve and are not in love with real time commenting, so we won’t be entering the ring ourselves. We have some money to donate to either wpDisquz or Epoch for immediate performance improvements: we’ll publish an update if either plugin gets much needed performance improvements.
At Foliovision, we prefer fast loading sites without too much complicated javascript (we spend our days and nights fixing AJAX and js issues on other sites) so we’ll stick to native comments with simple front end moderation from our Thoughtful Comments (which includes comment caching now, for real scalability on high comment volume sites). Performance before bling.
Martin Viceník
Martin graduated as an engineer in Computer Science from Slovak Technical University in Bratislava. He grew up in Liptovský Mikuláš in northern Slovakia next to the beautiful Tatra mountains. He is the developer behind our FV Player.
Thanks for the review of Epoch! We’re about to push a nice update to it in the next few days which has a focus on performance improvements. You can follow along (or jump in) at github.com/postmatic/epoch/issues?q=is%3Aopen+is%3Aissue+milestone%3A1.0.8
We have some interesting cacheing solutions in this release which should solve performance issues across the board. It’ll also fix the bug of Epoch loading itself when not needed (comments closed, etc).
What this release will not solve, however, is performance on posts that have more than 100 or so comments having a long page render in some browsers. That one is going to require we get moving on rolling out pagination support and/or a js viewport that only loads the comments you need to see (think imgur galleries). That and a rebuild based on the REST api are queued up for 2.0.
Hi Jason,
Thanks for the details on your ongoing development.
The people who need Epoch most are publishers who enjoy hundreds of comments per post. The sooner the better for large scale performance improvements. As I mentioned, we have some comment caching code in Thoughtful Comments which you might want to have a look at.
I’d put performance before features, considering the audience for Epoch (popular publishers). You might consider multiple configurations (shared hosting, small VPS) and dedicated server as they are very different environments.
We look forward to seeing the new version!
With 1.0 we were focused on can it be done more than making a solution for large publishers. We were hoping to create something that is a good fit for small to medium sites and, more importantly, get some momentum and excitement moving.
To that end I feel we have been successful: Epoch has turned into a mature open source project with contributions from across the WordPress world and a growing list of supporters. That’s more than I could have hoped for when we started out.
Our 1.0.8 release will make it a good fit for even larger sites such as those that do get a ton of traffic but less than 100 or so comments per post. Up next in 2.0 will be solutions beyond that…..
Best, Jason
WoW!!! Thant’s perfect. We were waiting for such article and we got it. Thank you very much Martin! This is a really great research.
I just want to say if you could wait for 3-4 days you’ll write totally different article. We’re happy to announce that in few days we’re releasing “wpDiscuz 3.0 Supercharged Comments” version. This is absolutely different plugin.
Please wait for a few days and I hope you’ll get tons of fantastic arguments to change or write another article abut WP Comments plugins.
— Thank you wpDiscuz Developers gVectors Team
Hi wpDisquz people,
Great to hear.
We could definitely use those new features and have wpDisquz in play on some larger and very active client sites. Please tell us when these features are launched so we can test them and send you some $$ support.
Making the web work for you, Alec
PS. Could you be kind enough to sign your comments in the future? It’s so much easier to talk to a person than “a team”. Trying to seem like Microsoft as smaller boutique companies plays against our strengths (the personal).
Hi Alec, Sure… I’m Tom. And thank you so much for your interest in wpDiscuz plugin and for your willingness to support it. We’re currently testing the 3.0 version and doing our best to release it by end of this week. We’re testing wpDiscuz with 10-50k comments on each post, the parent comments have 5-10 dept replies. The native WordPress comment system in most cases can’t work even with 1000 comments. If it doesn’t stop loading it loads those within dozens minutes. wpDiscuz 3.0 really supercharges the native comment system and it removes all limits related to comment counts and website high activity…
Hey wpDiscuz,
Glad to see you all keeping up the good fight. We’re big fans of what you are doing and love watching the evolution of your plugin. Native commenting FTW!
Jason
Hi Tom,
That sounds wonderful. Let us know as soon as we can download and help with the stress testing. I promise you very detailed feedback.
Hi Jason,
We’re looking forward to the next revision of Epoch too. For these sites though we need to be able handle high comment counts.
Thanks!
Thank you Jason and Alec, We relay appreciate your kind attitude and support. We keep supporting Postmatic in wpDiscuz 3 and relay like it. I see the reason of creating Epoch, and it’s very good developed, but like wpDiscuz it has lots of things which should be made better. So keep updating and lets bring people from 3rd party comment systems to native WordPress. The wpDiscuz 2 has lots of very useful features and it faster than 3rd party comment systems, but the performance and CDN compatibility was not good. So the main advantages of the new wpDiscuz 3 release are performance, speed on 100 and more comments (the real speed diff you can see on 100, 1000 or more comments), cache/cdn, live update optimization and super responsiveness on diff devices. i’ll be in touch very soon. It seems everything is going well with testing…
Hi, just want to let you know that wpDiscuz 3 is released. We’re on very active supporting stage… Detailed changelog you can find here: wpdiscuz.com/wpdiscuz-3-0-0/
Thank you!
Hey Tom,
Thanks! And, you’re welcome. We think wpDiscuz is great as well…. and agree that yes, there is room for all of us in this space. We see wpDiscuz as a great fit for sites that want an all-in-one native commenting solution, and then have Epoch for folks that want things to be more piecemeal. Let’s keep at it and get comments back where they belong.
Keep up the great work! Jason
Hello Tom,
we just added a part about wpDiscuz 3: foliovision.com/2015/10/disqus-wpdisquz-vs-epoch#wpdiscuz-3
Thanks, Martin
Hi Martin, Thank you for testing it again, it seems you’ve tested the auto-optimizer SQLs. It Works only one time on first page load. So please make sure this are correct data, it’s real important, and make sure you’ve tested 3.0.3 version.
And it seems you’re not see what’s happening here:
Here are more info about SQLs: wpdiscuz.com/wpdiscuz-3-0-x-sql-queries/
And here is an example how wpDIscuz 3 works very fast on website with 3.5+ million comments: wordpress.org/support/topic/suggest-ondemand-comments-load#post-7630702
Ok, I see the issue. You’ve tested wpDiscuz 3 without optimizing comments. So in other word you’ve not tested wpDiscuz 3. Please do another one or just correct the text in post. wpDiscuz 3 caches full parent/child relationship on first load. it may be x2 slower and with thousands of SQLs. BUT this is only on first time, that’s it. So you should check the second load of comments and you’ll see the correct result. You’re currently talking about comment optimization process SQLs and Speed.
You’re saying:
What’s the mean to report wpDiscuz 3 testing result without using the wpDiscuz 3 comment caching/optimizing system? That’s obviose that you’ll get such a bad result….
Hope you see what I mean and you’ll change those result, because those are not correct for wpDiscuz 3.
Thank you for you time.
Hi Tom,
Thanks for the testing notes. We’ll be sure to add your additional steps next week. I’d suggest making the process easier for your users with clearer explanations.
Thanks!
And some good news. wpDiscuz 3.0.4 is released with comments Lazy Load function. So it loads comments when you scroll page to down. That way you only load the post header and content. The comment section doesn’t use time and memory. It boosts performance delaying loading of comments in long web pages because comments outside of viewport (visible part of web page) won’t be loaded until the user scrolls to them. This reduce memory usage, page loading time and SQL queries. On a page load wpDiscuz only runs plugin settings initiator SQLs, those are about 5.
Hi Tom,
Thanks for the update.
I’m quite worried about the Lazy Load function. Comments play a significant role in Google rankings (we’ve tested for some of our clients). I think they in particular trigger the recency bonus. If wpDisquz 3 stops Google bot from reaching and indexing all comments, you may be doing your publishers’ sites a great disservice.
A more search friendly approach would be not to delay the loading of the body of the page yet load the comments later regardless of scrolling.
Warning: Undefined array key 1 in /srv/users/fvision-staging/apps/staging-foliovision/public/site/wp-content/plugins/thoughtful-comments/fv-thoughtful-comments.php on line 1449
Warning: Undefined array key 1 in /srv/users/fvision-staging/apps/staging-foliovision/public/site/wp-content/plugins/thoughtful-comments/fv-thoughtful-comments.php on line 1449
Warning: Undefined array key 1 in /srv/users/fvision-staging/apps/staging-foliovision/public/site/wp-content/plugins/thoughtful-comments/fv-thoughtful-comments.php on line 1450
Warning: Undefined array key 1 in /srv/users/fvision-staging/apps/staging-foliovision/public/site/wp-content/plugins/thoughtful-comments/fv-thoughtful-comments.php on line 1449
Warning: Undefined array key 1 in /srv/users/fvision-staging/apps/staging-foliovision/public/site/wp-content/plugins/thoughtful-comments/fv-thoughtful-comments.php on line 1449
Warning: Undefined array key 1 in /srv/users/fvision-staging/apps/staging-foliovision/public/site/wp-content/plugins/thoughtful-comments/fv-thoughtful-comments.php on line 1450
Hi Alec, Sure, we know about that, and we use AJAX Crawling Technology for our ajax loaded contents, this works for Lazy Load as well. If you inspect page source you can find such link: example.com/#!loadmore&parentId= xxx
More info here: support.google.com/webmasters/answer/174992?hl=en
So you should not worry about comment indexing issue.
Good to hear Tom! Thanks for the interesting reference. Have you field tested this (searching for exact phrases which appear way down the page)?
The AJAX Crawling URLs are the same for Lazy Load and for other load more buttons. We’re releasing 3.0.5 within next 24 hours with stable SEO features: Here an example how wpDiscuz loads ajax contents for Search Engine bots (this is the same for Lazy Load):
Video: screencast.com/t/CXrrTIpCiE3D
Hello Tom,
just a quick note before I get to check this in detail –
I was testing wpDiscuz 3.0.3. I tried to reload the same post over and over again, but I was getting no speed improvement and the SQL count didn’t lowered. Perhaps the built-in SQL optimization didn’t work for us.
Thanks, Martin
Hi Martin, Thank you for your time and the hard work you do testing wpDiscuz. We really appriciate it. Here some info that should used to see the correct advantage of wpDiscuz 3:
The main idea you should know that, native WP and wpDiscuz 2 get all 1000 comments using one SQl, if your post has 10,000 comments it’ll try to get it using a one SELECT SQL. So the number of SQL-s are 1 instead of 1000 or 10,000. This logic doesn’t work if you have many comments. it’ll stop loading comments. The memory of hosting service can’t store thousands comment on one page load and the MySQL buffer limits will not allow you get data. It’s just a matter of fortune that the native WP and wpDiscuz 2 on your server can load 1000 comments. Believe me about 90% cases WP and wpDiscuz 2 stop loading. And you can see that 1 SQL is hundreds of times can be heavy than small 2000 SQLs. So I recommend don’t report wpDIscuz 3 performance based on SQLs.
wpDiscuz 3 doesn’t use this logic, it only gets parent comments of current displayed comment threads and use very light and fast SQL to get hierarchy data. This works for each comment. There may be 1-3 SQLs per comment, it depends on enabled features. This SQL takes 0.0001-0.0003 second. 1000 x 0.0003 = 0.3 sec for 1000 comments. Please see this screenshot: screencast.com/t/wZgxgPj6
And ATTENTION!!!) It doesn’t matter how many comments you have for certain post. WP native system SQL get ALL comments, even if it shows 1 comment on front-end (paginated). So if you have 10,000 comments for one post, and you show 10 comments on first load, wpDiscuz 3 spends 0.0003 x 10 (SQL) = 0.003 seconds. But WP Native comment system will use the same resource (mysql time and memory): screencast.com/t/kdXYzn63X about 0.1 sec.
Nobody set 10,000 comment at once. The maximum number of loaded comments can be 100. So wpDiscuz 3 with 0.03 sec query time will always save your server resources. but the native WP will always operate 10,000 comments in DB. Even I’m not talking about big difference of memory usage.
P.S.: Please, I’d like to ask you to track SQLs like I mentioned above. Do not show SQL numbers, those show reversed result. Those don’t show the correct advantage of wpDiscuz 3 over native comment system and v2.
Hello Tom,
I tried that again with version 3.0.7 and no luck either. I logged all the SQL queries, but I don’t see any inserts of wpdiscuz_child_ids into wp_commentmeta. We tested this on WP Engine host in staging environment. I did nothing to break the “comment caching/optimizing system”.
We have some other website where we are using wpDiscuz 3 for around a week and I can see some wpdiscuz_child_ids on that one – so perhaps it works there. However the post with the most comments is just as slow as it was in wpDiscuz 2.
I checked your video with console as well. In our case loading 500 comments via Ajax takes on average 6.3 seconds, which is at least twice as slow as your loading of 900 comments in 4 seconds. I’m only checking the SQL query counts to understand why the load times are not improved.
Don’t get me wrong, I would like this to work and we really need it to work as the website where we use this will be getting the most traffic (10x as much) in December, so we really need something fast.
I also tested the Lazy loading of comments and I think this is the part which improved the most together with the “Load more comments” performance. I tried setting it to 5 parent comments per page and then it worked fine – the commenters here use a lot of nested replies. So clicking “Load More Comments” button typically loads 100 more comments in 2 seconds.
Thanks, Martin
Thank you Martin, I’ve posted comment a week ago here. But for some reason it’s not published. I just want to say thank you for your time. And ask you to change this post content related to wpDiscuz. Because that’s not actual for current 3.0.9 version. Since wpDiscuz 3.0.8, all SQLs have been totally optimized. And its even more fast and light. It comes with 39 SQLs instead of 2000 like it mentioned here. And the 39 are for unlimited number of comments.
Hello Tom,
sorry about the delay, I was meaning to get back to you much earlier. I’m not sure what could happen to your comment, we got no notification.
We tested wpDiscuz 3.0.9 on a clean Linode with a clean WordPress and no plugins. We only imported a single post with 801 comments and associated user account to test on.
Here we compared the resulting HTML page size, number of SQL queries, PHP processing time and maximum memory usage:
100 comments per page (we are talking about actual number of comments that shows up, as the WordPress setting determines the number of top level comments rather than number of comment that actually show up, so we tweaked that setting until we were getting 100 comments on our test page)
Great! Thank you very much Martin. As far as I see the huge change in the number of SQLs is already in testing result. The only reason why we don’t see the real speed is the cache. We’ve made some changes to make it more compatible with caching systems. And it’s already available in 3.1.0 > versions.
Also we should not forget that wpDiscuz has +40 features. And it’s not x40 times slower than the native. So you can see how fast is the core that it can load +40 features with approximately the same speed as the native. However It should already even be faster.
Hi Martin and Alec, thank you very much for the article with details and the benchmark data, it is very helpful. I am a developer working on a travel/tourist blog website preparing to process big data. I was looking for a chat solution and your articles really inspired me. I have made some changes to Epoch to interact with dynamic html code efficiently generated from mysql directly, in the wrapped template expected by Epoch. With the help of some custom fields and custom built index, the overall performance has been greatly improved. Here is a demo page to test the loading speed from over 150k comments nested under the same post (which are bible translations in public domain for users to quote without copyright concerns, each verse is one comment). There is a slight delay in displaying 1,000 comments per page, but 100-200 comments per page can be handled without difficulty. The comments are loaded at the same time of generating the web page, not through ajax post method, and thus is easy to cache such as using browser cache.
tagnmap.com/wp/photo/the-holy-bible/ Just append ?limit=200 (number of root comments) to the url if you would like to test how it performs. The comments (root and children) are kept at 1000 per page.
The website is currently hosted on Amazon t2 micro (VPS with 1gb memory), and this solution works on VPS and above. The comments are generated dynamically with user custom information (voted comments, bookmarks, likes, etc) , and is close to real time chat with user dynamics.
I believe with some fine tuning the solution can be a candidate for busy commercial websites. Please get in touch if some of your clients might be interested.
Hi Jason, not sure you will ever come back to this post. Nice work on the Epoch framework. Do you have plan of making a premium version of Epoch? I will try to get in touch with you to catch up.
best regards.
Oliver
Hi Oliver,
Your real time chat commenting project is fascinating. If you can keep the load down both on the client side and the server side. Even Disqus offers execrable performance. We ended up going a different direction and built more caching and like/dislike buttons into our Thoughtful Comments which now loads like lightning (20 ms) even with hundreds of comments.
Performance oriented websites should be looking more at comment caching than comment features which for the most part are lipstick on a site. I know there are exceptions and I’m not underestimating the value of community but fast loading pages (with lots of comments) build an audience and a community too.
Please post another progress report when you are ready though. Once it’s working well enough for commercial deployment, we definitely have a client who would be interested.
Hi Alec,
Thanks for the prompt feedback. You have made an excellent point in caching comments and reducing workload. After posting the previous comment, I studied your other article on comment caching and started to make further improvements to my solution by generating and storing caching pages at the server side, and have made some good progresses.
As for the storage of comment cache pages, I took an approach different from Thoughtful Comments in that the cached comments are stored as a custom post type, and are divided into separate posts if there are too many comments to show in one page (such as the bible verse demo page). This is mainly out of consideration for integration with indexing, updating comment cache in distributed computing environment and load balancing if the client’s site gets really popular.
Currently there are two cache versions, one for non-logged in users, and one for logged in users who have not commented or voted on the post. The difference is logged in users will have the links to reply comments and vote (thumbs-up) and can participate in the discussion whenever they like. when a post is requested, the script will first check whether the requester has logged in and has commented/voted on the post, if not, then the server will serve the post content with cached comments. The cached comments (which is actually stored as a separate post) are treated the same way as the post content, and can be cached on server side and client side. If the custom post of cached comments does not exist, it will be created on the fly saved the content in db for subsequent visits, and served to the requester. the cached comments are already in html format ready for browsers to display without parsing.
The total number of approved comments and mapping of search query to cached comment post are kept in memory, and the cached comment post will be updated at preset interval if the total number of approved comments have changed. If the post has closed for comments, the cached comments post will remain the same, saving valuable resources for logged in users active on the current post.
On my test set up, it takes less than 1 ms to find out the ID of the custom post containing the cached comments directly from memory without hitting the database or harddisk, and less than 1 ms to retrieve the post from WordPress db cache. so this should help to reduce page loading time. This set up uses some db or disk storage space, and some memory to hold the id/query variable mapping, and CPU usage is quite low.
New comments are loaded via ajax request as Epoch team originally designed.
It is relatively easy to build additional filters to show specific comments, in different views (timeline or nested tree), or comments within the last hour/day, or pending comments awaiting approval. I have built two filters for the demo page (bible translation and verse number, so when users click on the link, it will show the requested version or verse only, without leaving the page. The filters are processed using values already in memory without querying the db, and is very efficient too.
Currently the solution is customised for my project and hardcoded for efficiency. If there is a real need for this solution, I can separate the code from the project and create a premium WordPress plugin. I have sent an email to Epoch team to understand their development focus and direction.
The test server was working fine on Saturday, but since Sunday there were some connectivity issues which may have been caused by an unfinished cron job on the VPS, or interruption to Amazon services in general. When the server could be reached, sometimes the page with cached comments could be loaded between 3s to 4s displaying 1000 comments per page, while a few other plugins ( WooCommerce, JetPack, Pods) are running at the same time. I will do some further testing and report back the progress after hearing from Epoch team.
The comments with user information and indexing stay in the website which is an advantage over third party commenting solutions.
Thanks again for sharing your knowledge on comment caching, and tips on performance.
best regards .
Oliver
That’s a wickedly cool demo, Oliver. I’ll follow up with you via Github. Epoch is alive and well… twitter.com/gopostmatic/status/722422956748615680
Very interesting discussion, especially in the comments. I just wanted to not that when you said you ran the tests on WP Engine staging server, that AFAIK, WP Engine staging servers don’t cache.
Also, as I don’t deal with high-volume comment situations (yet), the main issue is UI/UX, not performance. Disqus, IMO, provides a pretty good overall user-experience. That said, I’d much rather stay on-site/native, so if wpDiscuz ends up close enough in the OVERALL experience to Disqus, you’ve probably won me over. (I’m hoping to test it on one of my sites in the next week or so.)
Hi Steve.
Our own Thoughtful Comments will end up being a Disqus replacement with better performance, a nicer look (very close to Disqus) and an easier to use interface. If you’d like to beta the pro edition, please drop me a line.