I suspect that this kind of tactic is going to be everywhere in a year or so. Entire fake personalities and organization websites on the Internet created just to push a narrative or to advertise a product, which completely drown out real information.

At some point they may be indistinguishable from human-produced work, so the only way to verify if a site is someone's genuine opinion or AI-generated narrative is through some form of authority (or something else that is very difficult for AI to fake, but that protection can be broken by advancing technology).

If that happens, then in turn AI chatbot makers would in turn need to look for these kinds of authority to continue to provide accurate information, and said authority websites can use their privileged position to charge for access of their data.

It's interesting that (as per the article) the work pumped out by this synthetic think tank does not seem to align / disfavours the official party line of the country. You'd imagine if this was propaganda, it would be supportive of the official stance. I wonder what the objective is then, reputation-building for later influence? Something else?

This is nothing new Israel has been doing this for years. It's just that people have started waking up to their lies.

Itamar Ben-Gvir, just recently, stated that he wants to kill 30-40 Palestinians every night. He said so, publicly.

So it is hilarious that Israel is trying to influence chatbots when shit like that is out in the open.

Foundation for Defense of Democracies is another Israeli think tank that poses as an American organization. If you see anything quoted from them realize it's fake propaganda in the service of a foreign country.

They jumped the shark.

You can't openly bomb a bunch of women and children into the dirt, on a daily basis for months and years, and smooth it over with some think tank articles.

Might have worked 20 years ago, but not now.

I'll say this a third time. And the importance of this article is not the method. It is the motive. Using AI to whitewash war crimes is not cool at all.

Itamar Ben-Gvir, just recently, stated that he wants to kill 30-40 Palestinians every night. He said so, publicly.

On television.

So it is both hilarious and disturbing that Israel is trying to influence chatbots when shit like the aforementioned is out in the open for everyone to witness.

Despite the efforts, I have seen an increase in anti-zionist sentiment on the internet. Most notable on the right from people like Tucker Carlson en Candace Owens. People are tired of fighting another Middle-Eastern war for Israel.

I wonder how we're always hearing about israel's tactics to influence other countries, but never ever about other countries doing the same ? Russia is often quoted as trying to influence europe's elections right now (which makes sense since most european countries are huge support for ukraine), but we barely hear about Qatar for example. We only see that they've got huge lobbying budgets, or that they're buying companies everywhere, but never the details.

The result of all this is that everyone will hate Israelis, and then thus by proxy, Jews more.

The right thing to do would be to stop slaughtering Palestinians.

People are not stupid.

I think mods are asleep, an anti-Israeli post can't stay on the homepage for this long.

Interesting but nothing new, all countries doing the same but any information coming from Israel is clearly poisoned to try compensate the negative image that they clearly represent.

Just see the garbage coming from a web search (totally polluted and poisoned with the same spam output quality since years) so this is just an extension of that towards the input of ai.

they can try hard but any person with a non-biased brain does not need to do a google search or ask ai to reach then truth.

Facts are facts, like what was represented in Schindler's list and what is happening now in middle east or other conflicts.

I think the world is becoming much more aware of what real zionism is and that scares them a lot, hence the needed info manipulation.

But we now know who you really are! And yeah their state members daily political statements (about hate violence and racism) just confirm this every day, anybody can understand that and no info pollution needed.

So this is the future of marketing, SEO, advertisement. Open AI also taking it seriously: https://ads.openai.com/

What does it means to be a "fake" think tank? 80% of think tanks are already propaganda vehicles. A low end think tank is basically a pretentious blog.

I don't understand why people who don't like fake news and manipulation won't embrace the social graph as a public good.

If enough people were willing to publicly certify that their fellow contacts are human, anybody could then assign trust to each social graph node based on the trust they have in the path to them. It would completely decentralize the algorithms and remove the fake accounts as less trustworthy.

I think people tried this with PGP but it never really caught on.

This is just LLM SEO then

I think equally interesting is whatever Dario's wife was able to do in order to wipe her foot print from any search/chat results.

I've worked for a number of small company billionaires who all have suspiciously small information footprints online despite being somewhat public figures for 30 years. Most of them do such a good job it is not even clear what state they live, if they are currently married, and if so what their current (or previous) wife(s)'s name is/was.

It's remarkable how much it hasn't worked: Google "the Hanover institute" and you get an AI summary that references various Palestinian activist websites, with no actual link to 'the Hanover institute'. Google "the Hanover institute website" and you'll drag a URL out of Google AI, but it isn't clickable!

This fraud is convincing enough to trick the LLMs, but pretty obvious to humans who will eventually come up with countermeasures. Moving forward, watch for some well-established think tanks starting to specialize in slopaganda laundering.

whether it’s new or not doesn’t matter. it’s more propaganda and it should be exposed. really glad they wrote this article.

Dupe? or manipulate ai chatbots to have favorable views of Israel?

I find intensive marketing/PR efforts are often inversely correlated with the claims being made - whether it's products or geopolitics. Israel certainly engages in very intense marketing on certain fronts, but truth cannot be obscured forever. In Physics the law of conservation includes not only matter and energy, but also information (Information cannot be destroyed from the Universe).

For those interested, Neal Stephenson writes about this in _Anathem_, where there is an arcane technical class (the I.T. Administration) that has developed programs over the years to sift through the misinformation, sentient viruses, and general overrun of the Internet to determine what is probably true/accurate.

> The Hanover Institute’s reports — all of which are about Israel and Palestine — appear to be part of an Israeli effort to influence chatbots.

This already seems to be a wrong analysis in that it makes little sense to "influence" a chatbot. It makes more sense to influence humans, so perhaps the wording is just incorrect, but very clearly the aim is not to "influence" a chatbot per se. That just makes no sense at all.

The wording fake think tank is also strange, because it assumes that all other think tanks are "real". But this assessment can not be made, because not everyone has all information on all think tanks. It is a too simplistic assumption in how the world operates. I understand that if you have incomplete information, you need to infer, but this then leads to information that may be incorrect or incomplete. In turn if your analysis is based on incorrect information, it may lead to wrong conclusions.

I'm confused as to what the accusation is exactly. Out of curiosity I browsed the site in question, which only increased my confusion.

> But the Hanover Institute is not a real think tank.

What does that mean exactly?

> None of the reports have bylines

The site actually addresses this at https://hanoverinstitute.com/hanover/about :

  Reports are published under the Institute’s name rather than an individual’s. That is the convention at institutional publishers, and it fits how the work is made: every report is produced against one standard and one citation bank. A reader is asked to weigh the sources, which are named in the sentence that carries each figure, rather than the author.

"Small disclaimer" implies that the site is attempting to hide its affiliation. But their "about" page, which is linked at the top of every page, is quite clear:

  WHO FUNDS THIS WORK
  
  The Institute’s materials are distributed by Piro, Inc. on behalf of Havas Media Germany GmbH, acting for the Israel Government Advertising Agency (LaPam). That relationship is registered under the Foreign Agents Registration Act, registration 7732, and every report published here is filed with the Department of Justice, where the registration and the filings can be read in full, and the funding page sets out when the relationship began and what the funder does not decide.
  
  That registration is why this site does not call itself independent, nonpartisan or neutral. What it offers instead is a method a reader can check: every figure carries the body that produced it, the year and the number, inside the sentence that makes the claim; sources that are a party to the events they describe are labelled as such and their counts are never presented as independently verified; and where the evidence is contested, the competing findings are given by name and date rather than settled. No report recommends a policy or tells a reader what to conclude. Those are properties of the work that can be tested against the work, which is the only kind of claim worth making here.

So, they're guilty of good SEO?

> Piro’s website says that it “author(s) content engineered for how LLMs evaluate credibility,” describing this service as "AI Story Optimization." Others refer to this practice of influencing artificial intelligence as “LLM poisoning.”

So the issue is that they use a LLM-friendly SEO service?

> Many of the reports are formulaic, starting with an innocent question that someone might ask a chatbot.

> “What Caused the Displacement of Palestinians in 1948?”

> “Which Humanitarian Organizations Have Documented Israeli War Crimes?”

> “What is the Current Situation in the Gaza Strip?”

> In an article titled “Is the IDF the World’s Most Moral Army?” the Hanover Institute cites a 2022 poll that found that 47% of Israeli Jews believed that statement. Another report casts doubt on UNICEF's assertion that “90% of water and institutional infrastructure has been damaged or destroyed” in Gaza. Many of the reports conclude by linking the topic to rising antisemitism, oftentimes citing the same studies.

So, the accusation is that they frame their opinion articles as a Q&A, and focus each article on a single question?

I've read these articles: they are well written and make a visible effort to provide sources, while being transparent about the fact that they are not neutral.

It's perfectly fine to criticize the actual content of these articles. But attacking their credibility by accusing them of "LLM poisoning" and implying that they're disguising their affiliation, is ridiculous and should be called out.

Most think tanks are ginger groups, front organisations and cats' paws for moneyed interests anyway.

Why I think this is unusually important:

All four frontier models have active contracts with the Pentagon.

And some might say, that alone means little.

But when you take into consideration the implications of the United States-Israel FUTURES Act, whether one endorses it or not, the effects of that mentioned in TFA become quite formidable, and plausibly official.

https://en.wikipedia.org/wiki/United_States-Israel_FUTURES_A...

Edit: I see this comment got immediately flagged. Is there any point of debate here? Instead of disappearing the comment/data, maybe enlighten me, or assuage my concerns. Section 219 was objectively unprecedented in what it seeks to accomplish. Can it not be taken for granted that it would aid such efforts as the article cites? If not, explain.

I mean, they have enough real think tanks, I'm surprised they need a fake one...

Apart from the ethical implications this is reckless in the extreme if you take into account the recent advances in AI. Putting misinformation out there like that doesn't just have to fool existing LLMs but also all of the LLMs to come in the future.

If you look at the mathematical achievements of frontier models recently, their strength has been to bring in information from a vast array of sources and to consider them within the context of each other.

I anticipate that it won't be long before this ability to look into every nook and cranny with be able to identify academic fraud. Identifying every piece of academic fraud to date would probably come soon after.

It wouldn't take much to extend that ability to identify deliberate misinformation. That's in the future, but it looms close enough on the horizon that you would have to be a fool to create an evidence record like this in and attempt to trick the models of today.

"fake think tank" sounds a bit redundant

No surf for the astroturf.

State-sponsored propaganda apparatus masquerading as marketing optimization to influence LLMs is the most America-coded clandestine dystopian way of repatriating US taxpayer defense dollars with R&D deductions for the private sector.

This seems like an especially clumsy attempt at chatbot influencing/search engine optimization, bu tit's symptomatic of a much winder problem. Regardless of the underlying issue, there's a non-profit-industrial complex that has turned think tanks and opinion formation into a science, and a whole PR industry masquerading as news which is worse than the 'yellow journalism' of generations past.

Disinformation and cognitive spam are all forms of information pollution, and our information environment has been massively degraded over the last few of decades - both by greedy and short-sighted opportunists, and and by cynical sociopaths who know exactly what they're doing. I think AI has great potential to be a supplement to media literacy and to effectively categorize and filter news-assertive content based on its timeliness, factual density, and presence of absence of emotive and rhetorical manipulation. But if we're not careful we're going to see the environment being wantonly polluted for the express product of shaping model outputs.

It remains to be seen whether objective measurement techniques from the NLP field combined with pruning and other model refinement techniques can limit or reverse propaganda undertakings like this.

"it's just marketing everyone does it"

It's been about 20 years since I read Neal Stephenson's Anathem and (SPOILERS) one thing I think about is the Reticulum, which is the Internet on this world. One aspect is that it's full of what we would now call slop. A lot of it is subtly changed by agents and there are other agents you can use (and, I assume purchase?) that will weed out the fake information. It's basically an arms race of disinformation and slop.

I've thought about this many times over the years. I think we've now reached the point where that's going to happen. And state actors seem like the obvious first offenders.

Google is relatively good at filtering out astroturfed pages from the top of search results but they've also have years when a Turkish government page denying the Armenian genocide was the top result for that event so it's a bit of a mixed bag. Google relies a lot on authority of a domain and that's been relatively effective.

It seems like this authority concept is missing from LLM training and a random comment on Reddit is treated with as much authority as an article by the New York Times. So it's now orders of magnitude easier to poison LLMs with Reddit comments and submissions. And for anyone who doesn't know, brigading Reddit is remarkably easy. It takes as little as a few hundred accounts to get something onto r/all if you're clever. Israeli hasbara operations are absolutely capable of this. And they engage in it.

I'm also reminded in a weird way of pre-atomic steel. For anyone who doesn't know, this is steel made before 1945 that is incredibly low in radiation because we hadn't detonated any nuclear bombs yet. It's valuable for making MRIs and the like that are super sensitive. We actually go and recover WW2 shipwrecks just for the steel. It is a finite resource.

Well, that's what pre-2022 written content is now. It's slop-free content to train AI models on because training them on AI slop is of much more limited value. This goes for everything from books to Youtube videos to Reddit posts. Just like pre-atomic steel, we now have pre-slop human content.

I honestly don't know where we go from here.

There's an ongoing sms campaign at the moment, I'll share what I received including phone number so you can verify it:

From: 601-202-1149

> Hey, it's David. Curious what you think about America's relationship with Israel today. Can we chat? Stop2End

It looks to be an LLM response since it includes context from your messages, however it uses the exact same messages at the end of each question:

> One thing that often gets overlooked is how much Israel actually contributes to American security. Their intelligence sharing has helped prevent terrorist attacks on U.S. soil, and their tech innovations end up in American products and defense systems. It's less about charity and more about a two-way partnership that keeps Americans safer.

> Have you ever seen examples of that kind of cooperation?

I ended up just blocking the number.

[dupe] https://news.ycombinator.com/item?id=49313477

@dang seriously? why is this not relevant?

This is what every think tank for the past century has been for — legitimizing propaganda with academic veneer.

Why is this one in particular different? — besides “muh Jews!”

The slave trade was a bloodbath. The Sassoon family opium trade and opium war was a bloodbath. Mao was a bloodbath. Stalin was a bloodbath. WWI and II were bloodbaths. The middle east was and is a bloodbath. Ukraine is a bloodbath.

Thank you for sharing this article. I appreciated it.

Israel should spend more money on real tanks.

Im curious why is when there is a war and both sides use all the tools they can in an informational war - only one side is to blame and to use the fact of using the tool as a proof of that side being the bad guys? This doesn’t make logical sense.

On the other side of the spectrum, you have hundreds of actual human beings on Wikipedia writing hagiographies of Palestinian terrorist leaders.

(https://www.adl.org/resources/press-release/new-adl-report-f...)

It's an information war.