Excerpt for Conception: Book One of Human Dilemma by , available in its entirety at Smashwords


Book One of Human Dilemma

by Scott Sibary

Copyright 2019 Scott Sibary

Smashwords edition

Cover design by Christopher Moisan

Thank you for downloading this ebook. This book remains the copyrighted property of the author, and may not be redistributed to others for commercial or non-commercial purposes. If you enjoyed this book, please encourage your friends to download their own copy from their favorite authorized retailer. Thank you for your support.

A minimum of 30% of the cost of this book (about 50% of the author royalties, varying with different retailers) will be donated to non-profit organizations that work to promote AI alignment with human values (such as those listed in back of this book. The first donee organization will be The Center for Human-Compatible Artificial Intelligence at UC Berkeley).

Disclaimer: Many of the names (forenames or surnames, never both) of the active characters in this book are coincidentally names of people known personally by the author. The use of those names in this book is solely for literary purposes. The active characters in this novel are fictitious, and any resemblance to real persons is unintentional and coincidental (unntatt, kan henda, Rolv). References to real or historical persons are intended to be accurate, and any inaccuracies are unintentional.

Table of Contents


Chapter 1

Chapter 2

Chapter 3

Chapter 4

Chapter 5

Chapter 6

Chapter 7

Chapter 8

Chapter 9

Chapter 10

Chapter 11

Chapter 12

Chapter 13

Chapter 14

Chapter 15

Chapter 16

Chapter 17

Chapter 18

Chapter 19

Chapter 20


About the author

Links to relevant organizations


Decisions, and decisions.

The day began with the clarity of a sky blown clear by north winds, autumn sunlight illuminating the west wall of her office, tea and coffee mugs steaming on the desk, and two minds fresh, buzzing and intent. Then, as the sunlight rose higher, the spotlight disappeared, the mugs stood empty, and the pace quieted. Yet the two persisted, driven by the career-changing proposal from the day before.

A computer screen on the desk displayed part of the code for a new set of algorithms she had dubbed “Protection Lock.” Its purpose was to guarantee the inalterability of the vital codes—codes aiming to ensure that an artificial general intelligence would remain altruistic, or at least benign. With completion of what the Chinese were calling the world’s first AGI scheduled for as soon as January 2039, Solveig had only a few months to prepare the Protection Lock for trial integration into that system, and only one month before leaving for China.

If she said yes.

Solveig Kleiveland leaned back, her reddish-blonde hair falling free behind her shoulders. Her well-worn cardigan with pewter buttons hung open. She’d been able to make just enough room for her slim figure to fit beside Lars at her desk without bumping their knees. Her younger colleague, in jeans and tight-fitting knit T-shirt, was similarly lean and as fit as a practitioner of modern dance.

“It has to be kept entirely confidential,” she said.

“None of it open code?”

“The Chinese are keeping the code for their immune system confidential. You, me, and the other team members from Norway will be the only ones with any access to the Protection Lock.”

“And you, Solveig, will be the only one with access to the part of the code that updates the Lock, assuming you’ve effectively secured the Lock itself?”

“Right. It’s enough that the vital codes will be open source, as they’ll have to be if the new World Council is to trust the AI. And that’s assuming enough nations continue to back the Indian Proposal, and the Council becomes a reality.”

“That’s the impression I got from the interview. Only, you haven’t yet agreed to go on the mission, right?”


They sat in silence. Solveig took long, deep breaths, eyes focused on infinity, like a distance runner who has completed one training lap and pauses to consider a second go.

Two soft raps sounded on the door.

“Come in!” she said.

Rune Andersen, the Director of the Institute for Humanistic Technology at the University of Bergen where Solveig and Lars worked, took a step inside the office. “Good morning!”

“Good morning,” both replied.

He closed the door behind him. His necktie and open sports jacket swayed left and right as he swaggered around to the opposite side of the desk. He raised his palms as if to beckon, mouth hinting at a smile.

“You’ve come for my answer?” Solveig asked.

“I need to ask you an additional question.”

“Wait, Rune. Don’t tell me . . .”

He raised his eyebrows.

She shook her head. “Don’t tell me you want me to be leader of the mission?”

He nodded.

“Why didn’t you mention it yesterday? Did someone else turn you down? Like Rolv Drammen?”

“No. The committee didn’t discuss the question until we were done with yesterday’s interviews.”

“Why not Reidar Hagerup? He and Rolv are older, and have loads of leadership experience.”

“Thirty-four isn't old enough? You’ve led research teams and taught classes, and it’s only a team of six.”

“Huh!” She coughed. “I’ve led one graduate seminar, and taught an introductory course one semester—not very successfully, in my opinion. And it was you who selected me to lead your small research teams. That’s hardly preparation for being the point person for a semi-secret international mission with potentially gargantuan impacts!”

“I’ll grant you that,” Andersen said. “Yet everyone on the committee, including the representative from the Foreign Ministry, thought you were the best choice. The design aspects of this project are more important than the relationships with the other scientists.”

“Or even with their leader?”

“Dr. Deng AnDe? Don’t worry about him. He has an excellent reputation for getting along with people, and he’s assembled a whole array of brilliant minds to work on their project. But he himself doesn’t have your technical genius.” He raised his hands questioningly. “Yesterday, Solveig, you seemed enthusiastic about the mission, open to anything. What’s your hesitation?”

Her focus moved from Andersen to the three photos on the wall. The first portrayed six middle-aged women wearing the traditional ceremonial dress of her home island of Osterøy. The feet of two of the women seemed to rest on a thick black line drawn on the photo with a marking pen. The second was a poster-sized aerial of Osterøy with the name of the hamlet of Hauge bright from a yellow highlighter, and farthest from the door hung a framed portrait of the Crown Princess of Norway.

“Things happen at night,” she said. “One gets shaken up.”

“Oh yeah.” Andersen shook his head and took a deep breath. “Don’t we all know.” He followed Solveig’s gaze to the first photo on the wall, rubbed his chin and then moved to look out the nearby window with a view over the Bergen harbor.

“Actually,” she said, and he swung back around.

Solveig glanced back and forth between the two men. “I don’t think I’d want to go if I weren’t the leader. The Protection Lock has to be handled correctly. And there are so many contingencies! What if the systems won’t merge?”

“That’s their problem. We just do our best.”

“What if they merge,” Solveig said, “but don’t honor the vital codes? Or their system penetrates the Protection Lock? Worse, what if we hand over our technology, but the World Council is never convened? We might be leaving the Chinese with a dangerous tool, maybe a weapon.”

Lars leaned back, one leg crossed over the other and angled towards her. He looked at her with a twisted smile. “Ah, so that’s it.”

“I have to make my own assessment of the risks,” she said. “I can’t make a knee-jerk reaction to go ahead just because the government deems it best. I bear responsibility, and so do you.”

He kept smiling as his hanging foot swung forward to tap her in the back of her calf.

“Lars!” She gave a tired laugh and shook her head. “I’m serious.”

“Of course. But your Protection Lock is useless by itself. What purpose can it have other than to play a role in a larger system?”

Which system? Whose system? That’s my worry.”

Lars threw up his hands. “Either the people on our troubled, shrinking planet cooperate more closely, or else . . .” He brought his fingertips together. “Tell me, can you prosper in a personal relationship if you don’t cooperate? The closer the relationship, the closer the cooperation. It has to be, right?”

“That’s different. Individuals always have choices—so many alternatives. You should find a good match, eventually.”

“Easy for you to say, you who ends things when you find something’s lacking. My situation is more like the international scene, with its limited number of nations. I don’t have many men to choose from, so I’ve embraced the spirit of compromise." He slapped the desktop with his hand. "And that’s why, as your closest colleague, I’ve told Dr. Andersen I’m willing to join the mission. And you?”

She leaned forward to rest her chin in her palm. In the quiet of the room, the buttons on her sweater swung against the edge of the desk with the resonance of a ticking clock, winding down.

“I share your concerns, Solveig,” the older academic said. “I think we all do.” He put both hands on the far edge of the desk, his shoulders jutting forward and his eyes beseeching. “Yet we need to know your answer. Will you be involved? Will you lead? It’s time to decide.”

The mind-deadening grey paint on the smooth concrete failed to hide the past. Those working in the old building of the Beijing health department knew what the main room of the basement had been used for. Empty since then, it would soon be given life. Thin, hollow walls had newly divided this subterranean world into several smaller rooms: conference rooms, a computer room, a guard room, a toilet room, and one larger, leftover area as a general meeting room that offered the only door in and out.

Dr. Deng AnDe stood alone, a tall but otherwise unremarkable statue of an everyday, forty-year-old man, and waited. The café where he’d just eaten a quick lunch had been playing the kind of dance music he liked: a fusion of swing and recent Chinese pop that had added to his energy and kept him moving fast, until he reached the basement. He glanced at his phone. He’d arrived twelve minutes early. He figured his boss, director of the Division for Artificial Intelligence within the Ministry of Technology, would probably arrive about one minute early.

The arrival of his boss should feel like a formal endowment of authority; AnDe could then feel in charge of this new job site. The transfer of jurisdiction over the basement facilities to the Ministry of Technology had been a mere technicality, but it symbolized a proactive step forward. At a time when many foreign governments remained as unstable as slowly spinning tops, his country would accept the leading role. It would, AnDe hoped, guide the world away from Armageddon and devastation, and safely into the future.

He could feel the weight on his shoulders.

He straightened his tie, carefully brushed down the front of his suit jacket with his palms, and took slow and deep breaths to let the tension flow out of his body. His feet began to wiggle in his shoes, and he felt his throat wanting to hum a tune. The quarter turn he took to look down the inner hallway resembled a dance step.

Facing the computer room where technicians were working, he again became a statue. In that room was assembled the hardware for the operating system of a new generation of artificial intelligence: an intelligence his teams would be creating with the participation of a small group from Norway.

He shook his head. How had that country managed to persuade China to cooperate on the Indian Proposal? India and the US had not. Norway’s influence seemed out of balance. Was it because the United Nations and the World Trade Organization had proven ineffective at facilitating significant international cooperation, beyond the treaty organizations they ran?

Perhaps his boss could throw more light on the question. It seemed to AnDe that the Chinese government had come to see international cooperation as necessary to prevent engineered disasters, like the influenza virus in 2029 that had killed nearly a billion people.

AnDe completed a scan of the site that would absorb the main focus of his energy for the next several months and was returning to the large meeting room when a familiar figure entered from the main building hallway. Director Liu, a man in his sixties, wore a casual business suit but walked as though he were in charge.

“Good morning, Division Director Liu,” AnDe said as his boss approached.

“Senior Project Manager Deng AnDe, thank you for offering me a tour of your new facilities.”

“With pleasure, sir.”

AnDe led him around the large room, and then paused to glance into the guard room. There, a uniformed watchman sat at a table below a bank of screens. The two continued past the three empty conference rooms and to the door of the computer room. That room bustled with activity. Several white-coated technicians were stringing wires, moving equipment, and testing circuits and routers. One person ran a vacuum cleaner around the corners of the room. AnDe and Liu looked through the door only long enough to receive a few over-the-shoulder glances, then turned away.

“You mentioned there was something you wanted my insight on,” the elder said.

“Yes, sir. There’s something I am curious—or you might say, confused—about. Our government had rejected the Indian Proposal and offered some criticism of the proposed structure of the World Council. But I don’t think that was the main concern. I put more weight on the distrust our government expressed about using an AGI as an official advisor. It seems to me that artificial intelligence has acquired a bad image: people in our country blame the pandemic on the AI that engineered the virus.”

“Yes, rather than blaming the dictator who thought it would help squash the drought-triggered rebellions in his country.”

“Or the arms producer who sold him the drones to deliver that plague, even if it was not supposed to be so deadly. Despite all that, our government now seems to be endorsing the use of AI.”

“About that I have only theories. When a nation is clearly the most powerful, it tends to resist any form of internationalism that it doesn’t control. We saw that with the US government on the subjects of arms control, climate change, and human rights. The plague, as some call it . . . it hit our country hard, but what were bumps and stumbles for us became a free fall for them.”

“And so their government fell to a coup.”

“Sort of. I suspect power was already being lost to private military contractors employed to quell the civil unrest. The coups by the traditional militaries in the US and UK restored a kind of government order, and they later handed power back to an elected government. But by then their economies had crumbled and China was the new economic leader.”

“That’s what I’m getting at. Once China became the most powerful, why did our government change its position and endorse a new international organization?”

“I wouldn’t call it a change. After we saw how much harm AI could cause—or I should say, could be involved in—it made sense not to pursue an internationalism that relied on AI. For the time being, that is. But much progress has been made on the design of vital codes. There are now several versions that might resolve the conundrum.”

The conundrum, AnDe reflected, was the great puzzle that if left unanswered too long could lead to the end of humanity. The more free-thinking the AGI, the more difficult it would be to direct. If too controlled, it would be too slow and limited to achieve significant new results; but if too dutifully focused on a particular result while remaining free in its choices, it would likely cause collateral harm. Yet regardless of the conundrum, the drive to create powerful AGI would not be stopped.

“Why endorse this version?”

“Ah, I’ve known you too long, AnDe, not to see there is something more you’re driving at. The Protection Lock?”

AnDe sighed. “Yes. We’re told it keeps the vital codes sacrosanct. We’re told it’s very strong, and in the future it could be readily transitioned for use in a quantum computer. But is there something else that makes this Protection Lock so attractive?”

“I don’t know, unless it’s simply that it works, in which case”—Liu held up his index finger—“it will protect the vital codes and guarantee that the conundrum remains answered.” He dropped his hand. “Or it may be just that those at the top have decided the proposal is a venture to investigate, to see what will work. They might like that the Norwegian modification to the proposal has the World Council sit in Beijing rather than Delhi.” He offered AnDe the smile of an insider.

A few paces from them stood a woman in custodian’s overalls, her back and one ear towards them, polishing two large, brass letters on a door: WC.

Liu looked from the door to AnDe. “Only one?”

AnDe sighed again. Might as well drop the subject and accept the answers one gets. “It’s unisex. Outside, by the elevators in the main hallway, are two more: a men’s and a women’s.”

The letters gleamed, and AnDe began to chuckle.

Liu stared at him.

“I apologize,” AnDe said. “I thought it humorous that the abbreviation could be the same for the World Council.”

The hand of the woman paused, and the squeaking of the cloth, still pressed against the letters, ceased.

“Surveillance?” he asked his boss.

Liu waived a dismissive hand in the direction of the janitor. “An old dilemma, control versus freedom. And here we engineers are trying to build an optimal middle path.”

“Don’t security officials understand that too much control leads to a decrease in productivity?” AnDe asked.

“Standardizes it, control does. Gets rid of rotten eggs but dims the stars. There are five stars in the Chinese flag, and we need all of them to shine together.”

“Yes. And with challenges like our worsening climate, the whole world needs all of its stars to shine, General.”

“General, yes.” Liu paused, as if reflecting. “Did you know I only reached major in the army engineers? They bumped me up in the Reserves. Because of loyalty, no doubt. Yet if they ever took me away from my position as director of the Division for Artificial Intelligence because they needed me full-time in the Reserves, it would already be too late. None of this would matter.” The slight gesture of his hands suggested he meant a world beyond the room in which they stood.

“That’s why I believe this mission is so important,” AnDe said.

“Youthful devotion,” Liu said. “That’s good. You will need it.”

Chapter One

What am I doing here?

She stood in the middle of a hospital ward, its rows of beds extending into the distance like a battlefield cemetery. Moaning rose around her like a dirge. She looked at the faces on the beds. None was distinctly recognizable, and they changed even as she stared at them: familiarity slipping away before it could be grasped.

A loaded gurney, sheet draped over the corpse, came straight at her. She took a step away, and it stopped beside her.

A hand shot out from under the sheet, grabbing her wrist. Then, like so many before, the scene faded away as her eyes struggled to open.

Solveig turned to the clock on the nightstand: half an hour before they should gather in the hotel lobby. She flew into the shower, the sweat on her forehead cooling her as she moved. Another nightmare to rinse away.

Half an hour later, she had finished an order of oatmeal from room service and stood in the middle of her room, holding an empty celadon tumbler in her hand. Her phone on the bed flashed and rang. It recited a message that the liaison had arrived and that her team was assembled and ready. She sent no reply. Instead, she raised her empty teacup to her face, as if it still held something of import: something that might decode a maze. Her fingers cycled the vessel over her palm while she examined a style admired for centuries.

This hard, transparent glaze exposes a world of underlying cracks, she thought. I wonder if I could decipher a pattern.

She rotated the cup in the opposite direction, as if a hidden pattern might reveal a clue to an intricate and mysterious insight: a metaphor for history.

She returned the cup to her breakfast tray and stepped over to the blackout curtains. Parting the heavy layers with her left hand, she faced the inclement weather, then stiffened from a renewed feeling of rebellion.

It’s all you have, the protesting voice insisted. What if the merger doesn’t work? You’ll be nothing, Solveig Kleiveland! You’ll be meaningless: a career wasted. And if it works the wrong way . . . Oh my god. It would be better to abort, or else you’ll be far, far worse than nothing.

Yet, the temptation! A chance to go beyond what they’re trying to do: how can you not?

She shifted her weight back and forth between anxious feet as a different concern ambushed her from its perch in the back of her mind. She’d let the turbulent currents of world affairs free her from a relationship that had gone stale. Like parting company with a temporary travel companion while disembarking at a busy airport, she’d simply let go of something that no longer had meaning. Yet in China, she’d just be getting older.

Forget that, Solveig. You’re not here for relationships.

She remained at the window, hoping to determine which of her objections was the greater culprit, or the greater sage, behind her doubts. As she sought clues, the weather merely reflected the question back at her: grey drizzle obscuring the view, sunlight diffused by an overcast. Such a scene could inspire any number of conclusions, driven by one’s inclinations. She stood rigid like an unwilling bride about to be wed to a groom she had never met. Outside was the groom: the giant metropolis of Beijing, with its size hidden by the mist and its power only hinted at by the glistening expanse of tall buildings. She had to wonder whether such a world genuinely wanted anything a poor country-girl from a small nation had to offer. And if it did, would that make it any better?

She knew this kind of cooperative project was the nature of her work. Her part would have no utility until integrated into the work of others—of strangers she would have to work with intimately. Strangers she was about to meet.

She closed her eyes.

“Sometimes it works out and sometimes it doesn’t,” her mother would mumble in a wispy breath. “You can’t rely on what you don’t know, but you can get a kick out of discovering!”

All kinds of kicks.

She passed her hand along the ribs under her left armpit: the kick in the side during her final rugby match, many years earlier, no longer painful.

Her classmates in school and college and her colleagues at work seemed to have no difficulty appearing affable face-to-face. In some it was clearly a put-on; in many it seemed genuine. Perhaps, she thought, some of them had never been betrayed.

The first betrayal, when she was in her first year of school, flashed into Solveig’s procrastinating mind. She had thought Helle in third year was being nice, introducing her friends. Solveig was as tall as them, as if she belonged. First there came the fake praise: Helle said Solveig was supposed to be very brainy, and so everyone should watch out. Then followed the ridicule: revealing that Solveig had never had a father, was poor, and her mum did grunt work because she was a sinner. Finally came the shove to get lost. Solveig struck back, but Helle was older and faster. She had allies. Afterwards Solveig couldn’t tell the full story and took all the blame.

A cloud had darkened the window of the hotel room. Solveig opened her eyes to rid her mind of the useless story and saw the reflection of a face gone from diffident to blank: a familiar look. Years earlier, a friend suggested she could look “as poker-faced as a Chinese diplomat.” Recalling the remark, she gave a laugh at the irony. She released the curtains from her long, thin fingers; it was time to go.

She held her head high while riding the escalator down to the bustling hotel lobby. An array of chandeliers came into view, to each attached seemingly countless cut-glass pendants: sharp, polished, and showing the world their brilliance.

She remembered what Andersen had said about Deng AnDe’s team, and steeled her gaze at the crystals.

The lobby décor was more Chinese than Western, reversing a design trend from the beginning of the century. The lobby echoed with the clatter of feet and the collective murmur of people coming and going. To one side sat a group of six men: her five Norwegian colleagues along with one Chinese man. Lars and Rolv she knew well; the other three she had just met in Oslo before their departure for China. The Norwegian men were visibly coordinated, each wearing a dark blazer, a light shirt with colorful tie, and dark slacks. Most were looking down, elbows on knees, not talking or looking around. She guessed they were embarrassed that their leader was not early, as the liaison from the Chinese Ministry of Technology had been. They were sitting under a chandelier, their downward-pointing faces looking dim in their own shadows, in contrast to the brightness above.

Upon seeing Solveig, the six men stood up. They watched her approach and seemed to be examining her clothing. There was similarity enough. She wore a dark navy-blue jacket with matching knee-length skirt, a light-blue silk blouse, sheer stockings, and pointed dress shoes. From her ears hung a pair of fine cut imitation gemstones matching the pale blue of her eyes, and around her long neck hung a silver chain with a circular Celtic pendant of entwined serpents. She had thought better of braiding her hair and left it simply gathered at the back.

They spent little time on greetings before the liaison escorted them to a waiting shuttle van. When they arrived at the basement of the old health department building, her team walked out of the elevator in wing formation. No gala reception awaited them, only a hallway dispenser of hot water for tea. As they approached the door to the new suite, she held her breath, stepped in front of the liaison, and strode in first. The liaison and Solveig’s team followed behind.

The Chinese team was standing ready in a neat arc facing them. The eight men and four women wore similarly styled business suits. The one exception was their leader, Deng AnDe, who looked somewhat less dapper. His suit was of a simple and mass-produced cut and a little baggy on his tall frame; the contrast with his subordinates gave Solveig a sense of relief.

He fashioned a broad smile that, judging by his warm eyes, she guessed could pass as genuine. She’d communicated with him before by text and voice messaging but never with any video attachments. Nor had she researched his personal background; she’d left that to the people working in security.

His arm reached out while bent in a nonchalant fashion. His eyes beamed excitement and his gait seemed sprightly in the context of a somber setting.

She caught herself shifting her weight to step back. She took one more step, planting her right foot solidly forward. She shook his hand and forced a smile.

The liaison approached to perform the formal introduction but Solveig spoke first.

“Dr. Solveig Kleiveland. Nice to meet you.”

“Senior Project Manager Deng AnDe; in English, most people call me ‘Andy.’ I have been looking forward to meeting you.”

First her grip relaxed, then her smile and her breathing. She took a step back.

Deng AnDe turned to Rolv, the tallest of the Norwegians.

“Senior Project Manager Deng AnDe. You must be Project Manager Rolv Drammen.”

“Yes, pleased to make your acquaintance.”

“Likewise. You’re also the deputy leader?”

“Correct. I help as I can.” Rolv proceeded to introduce his other colleagues.

The Chinese leader moved back to the center of the gathering. “After we finish our introductions,” he said, “I’ll give a brief welcoming speech. We can remain standing. Then I’ll give you a quick tour of the facilities. At a later date we will go across town to tour the facilities where my other teams work. I apologize there is no welcome banquet. Not my decision. Maybe today’s open schedule will give our Norwegian colleagues more time to adjust and recover for work tomorrow.”

And no mention of Q&A, thought Solveig.

Titles and names, titles and names: she had briefly studied them all from the file on the mission. Now they formed an unsteady collage of images and words that only blurred more as they were added to, until at last the introductions were over.

AnDe stood in front of the now-intermingled assembly to give his speech.

“Let me say again, welcome. This is very exciting. I hope you feel the same.” His look became serious. “All of us, around the globe, have been through something terrible. It showed us that we cannot continue with a disorganized world. Our leaders have decided to create a World Council as a new forum for international diplomacy and to coordinate on world problems, and they decided it should be assisted by a highly rational voice, to be called the World Electronic Analyst. The name was a result of compromise.” He flashed a quick smile.

“Our role over the next few months is to assemble that voice. We don’t know exactly how they’ll use it, but our artificial general intelligence will be the first in the world. The more the Council uses it, the more influential it will be. Maybe even . . . powerful. But if it’s powerful, it will also be dangerous, so how does humanity protect itself? What policies should form the fundamental values and imperatives of the artificial mind? As you know, experts from many disciplines have been working on this for decades. How should their answers be represented in the vital codes we are to install?” He shot questioning glances at attentive faces.

No one spoke.

“I think those are the most important policy questions in all of history,” he said. “Questions humanity must answer. Now we think we have good answers.” He faced the Norwegians.

“Our colleagues from Norway bring us the codes. They also bring their latest invention, the Protection Lock, to make sure the finalized vital codes are unalterable. Together, these sets of algorithms will make the Electronic Analyst safe and reliable. Most days two Norwegians will be joining each of our three Chinese teams. They will work with us, sharing their products with us.”

Not completely, thought Solveig.

“This is why I’m excited, because I’m confident we will succeed. We will have an influential, beneficial, and rational voice that will help our world leaders solve today’s problems. There will be no more holocausts, and our future will be promising.” He made a final check of expressions. “I think you all agree. Maybe it helps to reaffirm this hope as we welcome our guests. They make this project possible.”

Every Chinese face showed enthusiasm, though a few showed remnant traces of suffering. Solveig saw no enthusiasm on the faces of two of her Norwegian colleagues.

Chapter Two

It could have been any hour, her numb body told her. Solveig leaned away from the shuttle van window, craning her neck to read the dashboard clock, but her view was blocked by the pointing arm of the liaison sitting across from the driver. The van turned off the traffic-jammed street and headed down an alley.

Shortcuts, she thought. They’re too good at that.

The arm moved back, and she read 11:40. A deep voice caught her attention.

“I’ve just spoken with my wife,” Rolv said as they approached the hotel. “We’ll take our children with us to lunch in about an hour. They’re napping now.”

Beside him sat Reidar Hagerup. “That works for my wife and me, if you’d like the company.”

“Yes, please join us.” He glanced around at his colleagues. “You are all welcome.”

Per Nilsen, who worked on cybersecurity, faced Rolv with glazed eyes. “The jetlag is hitting me hard. I’m going to bed; I don’t care when I eat.”

Solveig found herself admiring his movie-star face and trim body, then quickly averted her eyes. She saw Lars had caught her and was smiling. She looked at Stig.

“Then three of us for lunch now?” she asked.

The red-haired giant with a tenor voice, offered an eager grin. “You bet,” he said.

The hostess in the hotel café led them to a table in the center of the room. Clinking and clanking and unfamiliar language rippled like a flooding stream around them. Solveig spotted a table in a quieter area near the far wall. She pointed at it. “What about that table? It’s not very private right here.”

“Nowhere is,” Stig said, projecting his voice to the room.

He and Lars sat down across from each other, leaving the chair between them for her.

Solveig glanced at the patient, unmoving expression of the hostess, then pulled out her chair and sat down.

The hostess pointed to a center console. “You can order from this screen or just use it to signal me.” She handed them westernized menus and left.

Stig’s wide smile shone. He looked from Solveig to Lars. “You two have known each other for how many years?”

“Seven,” Lars said.

“I read your professional backgrounds, but you don’t post much on social media, do you, Solveig? And there aren’t exactly books written by you either, Lars, though I saw you like dance. How did you get interested in computers?”

“I did more social media years ago,” Lars said, “but the Facebook scandal provoked me. I kind of woke up to the addictive aspects. So I studied data protection, and how invasion of privacy can lead to manipulation. Eventually I migrated into safeguards.”

“I get it, of course. Myself, it’s similar, wanting to mess with the systems I’m playing with. A true nerd at heart.”

He looked at Solveig. “Could I ask you the same question?”

She cleared her throat, then took a sip of water. “OK. You may already know that Lars and I met working on control protocols? I was a doctoral student and he an undergrad. Neither of us wants a powerful system unless it has proven safeguards built into it. And with humans able to guide and understand its actions.”

“Of course. Who doesn’t want control?” Stig asked.

“Full control isn’t our goal. You ought to understand that. We’re developing something with basic human values, so that it will come up with approaches that will satisfy us. Since it will tackle—maybe someday even comprehend—things we’re not smart enough to, any attempts at full control will only undermine its abilities.”

“I’m on board.”

“Many computer engineers put aside the concerns for safeguards.”

“Pressure to get the product selling,” Stig said. “Get it up and running and ready for distribution.”

“A different value judgment,” Lars said.

Stig nodded.

Their food arrived: a plate of curried vegetables for Solveig, fried shrimp and noodles for Stig, and a bowl of thick soup for Lars.

They ate quietly for a minute before Stig asked, “Either of you have hobbies?”

“Sure,” Lars said. “Dance, tennis, reading fiction. I could mention more.” He extended a palm towards Solveig.

“It’s been a changing mix in my case.” She checked their expressions; they waited. “Since I was a kid, I’ve enjoyed puzzles and riddles. Also math. I even like learning languages. I always had tons of chores to do, including repairing things—a lot of things. I enjoyed it and it gave me confidence.” She hesitated. “We were poor; most of what we owned came from thrift stores. I didn’t mess with electronics until my teens.”

She paused as she noticed Stig’s head turn to follow a shapely waitress in a tight dress. “I got fed up with the games and wanted to create my own.”

“You smashed your computer during one, didn’t you?” Lars said.

“Oh! Tell us about that!”

“You would bring that up, Lars.” She ate a couple more bites of her food. “Yeah, I was cornered one day in the alter-world, facing annihilation. My real arm twitched, and I could swear it acted on its own. It smashed the controls into the screen.”

“Wow, I wouldn’t have guessed you were the game-playing type at all.”

“Is that a criticism or a compliment?”

Stig shrugged.

“I prefer open games,” she said, “where you know what’s going on. Like sports, including tennis. Lars and I play. He’s better.” She coughed once. “You might say sports got me more involved in life.”


“In my mid-teens I was tall and fast. An acquaintance was setting up a girls’ rugby team, to join a local league. I don’t know, it’s one of those strange decisions you make, especially as a teenager. I joined, starting as a back. As I worked out and gained weight, they moved me to the scrum.”

“Bet you were heavy then.”

She put down her chopsticks to look straight at Stig. “Somewhat—heavy enough. I enjoyed the intimacy of the scrum: a new kind of experience, your arms around a teammate on either side, all pushing forward against the other team until one side gains the ball.”

“That’s what you see us doing here?”

“Definitely not,” she said.

“Then, does your story connect to computer science?”

“She got hurt,” said Lars.

“Yeah.” She aimed a stare at Lars, drew in a breath and continued. “I broke a clavicle and two ribs. I stayed home for a few weeks, playing computer games.”

“But you stopped after breaking your computer?” Stig asked.

“I couldn’t afford a new setup, so I went to the library, played instructional games, made up for lost time, and actually got way ahead of my equivalent school level. But it was there again, that queasy feeling I always got from computer games, that feeling of being guided by something I couldn’t see. And then the nausea of feeling controlled. Being shoved around in rugby is part of the game. And you shove back. You all know the rules. But I realized the one-on-one with a machine was controlled by hidden algorithms. The engineers had turned the tables to make me the object.” She rested her elbows on the table, stared into space and tapped her fingers against each other. “I became determined to figure it out.”

The other two paused their eating and waited.

“Only by lifting myself to that level, by knowing the ways of the sorcerers behind the screens, could I feel empowered. That became my first quest.”

“Then you should be interested in policy,” Stig said. “That’s where it all starts.”

“Back then I wasn’t. I hated my high school civics class, that exercise in torture. I remember spending an entire term arguing about policy—climate change policy—the problems dumped on us because of an irresponsible adult generation. Policy failure, over and over again.”

Lars raised his eyebrows.

“We’re the adults now,” Stig said.

“Exactly. And yes, policy concerns about how AI is managed got me working on vital codes. You too, right, Lars?”

“Right. I’ve worked on enabling the AI to consider all the repercussions of its decisions. Remember the issues with self-driving cars, or that old dilemma of a train on a collision course, whichever track it takes, and the AI is the only thing that can make a decision quickly enough. Only, real situations could be more complex than our little brains can grasp, and program for.”

Stig nodded. “And being obedient, following orders, isn’t enough instruction to include in the vital codes.”

“That was my point earlier,” Solveig said.

“Why only minimize harm? Why not avoid it altogether?”

“Because harm could include almost anything. You could finish that generous dish”—she pointed at his nearly empty plate—“or not, but there are consequences either way.”

He put down his fork. “So Lars, is that your focus within Solveig’s project at the university?”

“Yeah. Code flexibility: how the system can figure out what we’d want if we were able to fully understand the situation. All part of the vital codes. Solveig led the subgroup trying to make the vital codes unalterable by users, administrators, and even the machine itself as it self-programs.”

“And you succeeded!” Stig said to Solveig.

“No. My subgroup kept failing. Only after I got back from India did I strike upon the idea.”

Stig’s eyes widened, complementing his beaming smile. “Come on, tell me how.”

“I got out of bed in the middle of the night, after wrestling with a nightmare. I sat up doing sudoku puzzles, got bored, my mind drifted, and suddenly, I saw it!”

“The Protection Lock?”

“Yes.” She glanced around the room.

Purchase this book or download sample versions for your ebook reader.
(Pages 1-23 show above.)