Computing desk
< May 25	<< Apr \| May \| Jun >>	May 27 >

Welcome to the Wikipedia Computing Reference Desk Archives
The page you are currently viewing is an archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

May 26

Existing API(s) to read from and write to web pages (from Fortran) ?

Are there any ? If not Fortran, how about C ? I'm only talking about reading and writing a few characters per call. I'm interested both in being able to write to web pages that don't require passwords (like Wikipedia) and other sites that do. StuRat (talk) 19:06, 24 May 2011 (UTC)[reply]

I suspect that they exist; however, I think that they'll be a great deal of trouble to use. Fortran and C both suffer from being languages in which string manipulation is difficult (and that's a lot of the work that goes in to reading and writing web pages). As a result, libraries for those tasks may be (a) less-used and more poorly maintained, and (b) harder to write in the first place, so less full-featured. Scripting languages tend to be much easier to use for this purpose, and come with "batteries included", as they say.

So, with the caveat that doing this in C will probably be a frustrating experience, the curl people have a list of HTTP libraries. Paul (Stansifer) 06:34, 26 May 2011 (UTC)[reply]

Thanks. I agree that string manipulation is difficult in C, but not in Fortran. StuRat (talk) 08:20, 26 May 2011 (UTC)[reply]

DVD-R compared with DVD+R

My computer is a few years old. It writes things to DVD-R disks without any problem, but it finds many errors when verifying DVD+R disks. Does anyone know why please?

After reading the appropriate articles, DVD+R is the newer standard. Might my DVD drive be unaware of the DVD+R standard, or would it reject it even if the standard did not exist when it was manufactured? Thanks 2.101.10.94 (talk) 10:41, 26 May 2011 (UTC)[reply]

The drive will reject the disks it does not support. It may be possible to burn at a different speed, or to supply new microcode for the DVD drive. Graeme Bartlett (talk) 11:11, 26 May 2011 (UTC)[reply]

Older drives (say pre 2005) used to have problems with one format or the other, depending on the brand. I made use of slower writing speeds and that helped e.g. lower the default 12x to 8x. Newer drives don't seem to have a problem with either disk types or speeds. My 2009 Liteon works like a charm at 16x. These things are relatively cheap to replace. Sandman30s (talk) 06:20, 27 May 2011 (UTC)[reply]

internet protection & security

what is steganography? — Preceding unsigned comment added by 59.98.194.156 (talk) 12:11, 26 May 2011 (UTC)[reply]

See steganography. -- kainaw ™ 12:20, 26 May 2011 (UTC)[reply]

Ha, nice try! If you look carefully, the OP's question includes embedded in the word "steganography" the cryptic signal "no ra", clearly an allusion to Ra, the ancient Egyptian sun God. Most likely, "no Ra" is a signal for a sleeper cell to initiate a devastating program, whose details, of course, we cannot know; but the fact that the signal coincides with recent news about volcanic ash possibly grounding flights is too much to ask us to attribute to chance. No Ra, because the clouds are filled with ash. And no Ra, no flights. This terrorist signal concerns airplanes - a notable, and vulnerable, terrorist target - and therefore demands the utmost urgency! I would waste no time on beginning investigations into "59.98.194.156".94.27.220.223 (talk) 23:43, 26 May 2011 (UTC)[reply]

Dude, you should join up with Harold Camping. Combine his ability to somehow look at something ordinary and somehow draw a ridiculous conclusion with your own and you would be unstoppable--Jac16888 ^Talk 23:46, 26 May 2011 (UTC)[reply]

Yes, I do pride myself on my ability to find the little extra that shows that something "ordinary" is really ... unusual 94.27.220.223 (talk) 00:03, 27 May 2011 (UTC)[reply]

I was thinking more Alex Jones myself. ^Avic _ennasis @ 12:44, 24 Iyar 5771 / 28 May 2011 (UTC)

Fake images and computer graphics

This is a paper that I referenced in a pattern recognition paper that I wrote. I just checked and it is still online. It covers a lot of the algorithms used to easily detect altered images through early 2009. There are more advances in the last couple years, so they are getting even better. -- kainaw ™ 14:04, 26 May 2011 (UTC)[reply]

These all rely on the statistical character of the tampered and untampered parts of images being distinguishable. That's effective because image manipulation tools like Photoshop make no attempt to fool such analyses. If someone, or some organisation, that is technically proficient and cognizant of these techniques were to try, they could surely process the unaltered parts of an image such that these differences aren't evident. And the very images that those with sinister motives would most want to alter (e.g. security cam footage) are often the noisiest to begin with, all the better in which to hide manipulations. -- Finlay McWalter ☻ Talk 14:48, 26 May 2011 (UTC)[reply]

Good point, the common tools leave traces, because they are not designed to hide their effect from serious scrutiny. SemanticMantis (talk) 14:54, 26 May 2011 (UTC)[reply]

First, I can't tell whether you are interested in wholly synthetic rendered images, or images that are based on an original 'real' photograph, and I think the answers may vary a bit between the two. Kainaw's link addresses a forensic analysis angle, where a computer is tasked with distinguishing a 'real' JPEG from one which has been manipulated. This doesn't tell us much about human perception, especially humans who aren't skilled experts. Anecdotally, I have seen children be fooled by CGI in movies. So, while experts and computers will be able to distinguish synthetic/manipulated images from 'real' images for the near future at least, I think that CGI will be able to fool an 'average' human rather soon. SemanticMantis (talk) 14:54, 26 May 2011 (UTC)[reply]

It also depends on the subject. At the extreme end, I can produce a very accurate picture of a scene without any light sources. See Cornell Box (and an example side-by-side comparison) for another simple example. If you look at raytracing competitions, you'll occasionally see impressively photorealistic images, though it depends a great deal on the artist and the subject matter (for example subsurface scattering doesn't appear to be common yet, so people tend to look like they're made out of plastic). Paul (Stansifer) 15:50, 26 May 2011 (UTC)[reply]

Thank you folks for all your excellent answers. @SemanticMantis: I was interested in both wholly synthetic images and images that are actually altered 'real' photographs. I've yet to come across a rendered image that can fool human eyes, but the examples in the paper Kainaw linked above seem to suggest that those with sufficient knowledge and skills already can produce partially altered images that can fool an average human. Then, although it is a potentially dangerous tool at the hands of those with sinister motives (altering security cam footage, celebrity pictures, historical photographs, etc), it is also a fertile ground for artists who now can create images that are surreal and hyperreal at the same time, I suppose. --BorgQueen (talk) 16:29, 26 May 2011 (UTC)[reply]

For wholly synthetic images, there are two main challenges: (1) Textures. Even the best computer representations of things like skin, wood, grass, cloth, or hair still fall short of the complexity of reality. (2) Lighting. In a natural scene, every visible object contributes light to every other object, not to mention all the light that comes from things that cannot be seen. Even the best modern raytracing techniques are challenged to encompass all the possible light paths. Looie496 (talk) 23:49, 26 May 2011 (UTC)[reply]

How to edit Wikipedia from my Windows XP Command Prompt ?

You could probably do that with a php or cURL script run from the command line. I don't know enough about either of those to say exactly how, but many of the bots on Wikipedia are written in php and I've seen example cURL scrips which could post on other websites. The Wikipedia:Creating a bot page has some detailed information about writing scripts which can edit Wikipedia AvrillirvA (talk) 16:42, 26 May 2011 (UTC)[reply]

Thanks, but those seem more complex than I had hoped. I was looking for something like this:

AppendToWikiPage http://en.wikipedia.org/wiki/User_talk:StuRat "StuRat is on vacation."

It could either edit under my name, if I was currently logged in, or anonymously, otherwise. All the edit tokens and other stuff should be handled internally. Is there some reason why it's not possible to create such a basic command line function under the Windows XP Command Prompt ? StuRat (talk) 17:52, 26 May 2011 (UTC)[reply]

This doesn't answer the original question, but if you just want to automatically update your userpage with "StuRat is on vacation" without having to load a browser and make the edit and you know the date of your vacation, you could use template:Show by date to display the message after a set date. For example, "{{show by date|2011|05|27||StuRat is on vacation}}" would display the message tomorrow. AvrillirvA (talk) 19:14, 26 May 2011 (UTC)[reply]

You can implement that functionality using cURL, wrapped in a shell-script (or with Perl, or any other common scripting language). You will need to script functionality to deal with the specific implementation details of MediaWiki; for example, you need to specify which DOM element is the edit-box; and what action to perform on "submit". Such details rarely (or never) change, so they could be hard-coded in to your script; and your command-line need only specify what you expect to change. You should read about the Document Object Model before you attempt programmatic web-page editing. MediaWiki's edit-box and wiki-syntax are wrapped inside HTML, so you will need a bit of "boiler-plate" code to stuff your wiki-syntax into the page. Nimur (talk) 18:05, 26 May 2011 (UTC)[reply]

As pointed out above - you are getting into socket communications. You pointed out above that you do not want to deal with socket communications. So, if you avoid communicating on the network, it becomes very difficult to communicate over the network. Further, Wikipedia doesn't allow simple automated editing. You have to program your side to give Wikipedia the impression that your program is actually a web browser. That keeps the script kiddies who know nothing about socket programming from writing scripts to spam "KIDZ IS THE SUKKORZ" on every Wikipedia page with a single command. As for the "why hasn't somebody written this already?" - they have. There are hundreds of Wikipedia bots. Most of them are open-source and you can download/use them if you take the time to figure out how they work. -- kainaw ™ 18:06, 26 May 2011 (UTC)[reply]

Regarding "if you avoid communicating on the network, it becomes very difficult to communicate over the network", that's like saying "if you avoid building your own car, then you can't drive a car". I'm fine with all this socket programming happening, I just don't particularly want to start from scratch myself, and "reinvent the wheel". I am hoping this low-level stuff is already done, so I can spend my time on the part that's more interesting (to me). Similarly, by "if you take the time to figure out how they work" statement, if you mean I have to understand all the internal workings, then I would ask, why ? When I make a program, the users don't have to understand all the internal workings, they just need to know the inputs they provide and the outputs it produces. StuRat (talk) 18:39, 26 May 2011 (UTC)[reply]

Here's a tutorial from cURL to perform logins and form submissions. Using cURL to automate HTTP jobs. Everything StuRat has asked to do is explained in detail, with code examples, in that tutorial. You will need a cURL request to the Special:UserLogin page; you will need to store the login-cookie; and you will need to submit an HTTP post request, including your text submission, qualified with your login-cookie. Nimur (talk) 18:07, 26 May 2011 (UTC)[reply]

It can't edit as an anon I/P ? StuRat (talk) 03:30, 27 May 2011 (UTC)[reply]

I guess this probably isn't what you're looking for, but when I think of editing Wikipedia from a Command-line_interface, I think of Lynx_(web_browser). SemanticMantis (talk) 19:49, 26 May 2011 (UTC)[reply]

The same for me, as well. (Edit from Lynx :-D) Given what all I had to do to get this far, it seems highly unlikely one could reliably program an automatic edit in Lynx. ^Avic _ennasis @ 15:51, 23 Iyar 5771 / 27 May 2011 (UTC)

If Wikipedia puts blocks in the way of automatic updates, what other sites don't ? StuRat (talk) 18:46, 26 May 2011 (UTC)[reply]

If you have Python on your PC, you could use Pywikipediabot with the add text.py script. This would let you use something like

add_text.py -page:User_talk:StuRat -text:"StuRat is on vacation." -always

Not sure if that's what you're looking for or not. ^Avic _ennasis @ 16:01, 23 Iyar 5771 / 27 May 2011 (UTC)

(ec w.Avicennasis) It's trivial to update a MediaWiki wiki automatically with Python+mwclient:

Python

#!/usr/bin/python 

import mwclient,sys                     # need to install mwclient from http://sourceforge.net/projects/mwclient/
  
site = mwclient.Site('siteOfTheWiki')   # e.g. en.wikipedia.org
site.login('myusername', 'mypassword')  # my username and password; if you don't do this line, you'll edit as anon
page = site.Pages['pagetoedit']         # name of the page to edit

text = page.edit()

# one bullet item per argument; you probably want to wrap the args in quotes so the script sees only one arg
for x in sys.argv[1:]:
    text += u'\n* ' + x  

page.save(text, summary='some standard edit summary')

I've not tried this on a Wikimedia Wiki (just my local one), so I'm not 100% sure that one of Wikimedia's special protections won't throw a captcha or something that it can't do (and I don't know if using this here complies with the bot policy, even if its a little test page in your userpage). -- Finlay McWalter ☻ Talk 16:15, 27 May 2011 (UTC)[reply]

Wikipedia:Bot policy states: ...any bot or automated editing process that affects only the operator's or their own userspace (user page, user talk page, and subpages thereof), and which are not otherwise disruptive, may be run without prior approval. ^Avic _ennasis @ 18:27, 23 Iyar 5771 / 27 May 2011 (UTC)

Excellent ! ...I downloaded Pywikipediabot and Python 2.7.1 (apparently V3 is incompatible with Pywikipediabot). It works. Thanks ! StuRat (talk) 19:19, 27 May 2011 (UTC)[reply]

Get old data with cURL

I use the following cURL command to retrieve data:

curl http://en.wikipedia.org/wiki/User:StuRat/cURL

It worked great the first time. However, when I updated the web page, cURL continued to retrieve the old version. Is it finding it in the cache ? How can I tell it to ignore the cache and read directly from the web site ? StuRat (talk) 19:40, 26 May 2011 (UTC)[reply]

Yes, it's getting a cached version from the squid. Bypass the squid with --header Pragma:no-cache -- Finlay McWalter ☻ Talk 21:22, 26 May 2011 (UTC)[reply]

Excellent ! (BTW, what is "the squid" ?) StuRat (talk) 03:24, 27 May 2011 (UTC)[reply]

Squid proxy servers run by the Wikimedia Foundation cache fully rendered article pages to reduce the load on the main server. Logged-in users never get the cached pages, but curl didn't send a cookie with your login information, so you were treated as an anonymous reader. -- BenRG (talk) 05:36, 27 May 2011 (UTC)[reply]

Resolved